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ABSTRACT 

The purpose of this study was to determine whether 
the variables found to influence learning in laboratory settings were 
able to produce effective student achievement when utilized in 
Venezuelan distance education. Three versions of the same 
instructional content were employed, and 42 volunteer college 
students were randomly assigned to three experimental groups. Group 1 
received a text in which such instructional aids as objectives, 
advance organizers, and exercises wer<i interspersed with the reading 
material; Group 2 received the saine content, but the text and aids 
were separated; and Group 3 received only the text materials. A 
posttest-only design was employed. It was found that Group 1 
performed at a significantly higher level in the posttest than Group 
3. Organization, condensation, and content that meets the needs of an 
adult learner were found to be the factors in the study materials 
that were most effective for these students. There was also evidence 
that different versions stimulated different learning activities and 
that higher achievement was obtained when the instructional aids were 
taken into account. Although all students obtained the lowest scores 
on items requiring deep processing. Groups 1 and 2 obtained higher 
mean scores than Group 3; the deepest levels were attained by 
subjects who reported that they had applied the study techniques 
recommended in the instructional material. The text is supplemented 
by 14 tables, 1 figure, and 80 references. Three appendices provide 
copies of materials used in the study, and a fourth presents a 
commentary on the study's methodology by R. Schuemer together with a 
reply by the author. (Author/EW) 
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INTRODUCTION 



Anne Benko de Rotaeche's study as printed here is of particular interest 
to distance educators as it looks into the influence of various procedures 
applied to support learning from written texts. 

It is well known to distance educators that techniques have been developed 
to direct students' attention to important issues, to considering and 
searching for solutions. Various designs are used to help students to 
structure their learning. Apart from graphical methods such as headings, 
graphs, and change of type a number of pedagogical pointers are used. Here 
belong attention-directors of various kinds. Those which are most typical 
of distance-study courses are suggestions to note or review something and 
self-checking exercises. Assignments for submission can also have this 
function. 

Of general interest in this context are Ausubel's 'advance organisers' 
which 

'are introduced in advance of the learning material itself and are 
also presented at a higher level of abstraction, generality, and 
inclusiveness; and since the substantive content of a given organizer 
or series of organizers is selected on the basis of their 
appropriateness for explaining, integrating, and interrelating the 
material they precede, this strategy simultaneously satisfies the ... 
criteria ... for enhancing the organizational strength of cognitive 
structure. ' 

(Ausubel 1968 p. 148) 

Ausubel, who distinguishes advance organisers from summaries or overviews 
which 'are ordinarily presented at the same level of abstraction ... as 
the learning material itself (ibidem), argues in favour of a hierarchical 
theory of cognitive structure. New learning materials are seen as items 
which are subsumed unter already existing cognitive structures. Early 
research on the whole gave proof of the effectiveness of advance organisers, 
but later studies have produced conflicting evidence as to their effectiveness. 

The advance organisers describe the basic concepts of the immediately 
following part of the course and 'bridge the gap between what the learner 
already knows and what he needs to know before he can successfully learn 
the task at hand'. They have proved helpful to students because, 'not only 
is the new material rendered more familiar and potentially meaningful, but 
the most relevant ideational antecedents in cognitive structures are also 
selected and utilized in integrated fashion' (Ausubel 1968 pp. 148 and 137). 
Advance organisers thus, in Rothkopf's terminology, would seem to promote 
mathemagenic positive behaviour, that is behaviour that is conducive to 
individual learning (mathema = learning; gignesthai = be borne). 

9 
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The use of questions in the text is another procedure applied to structure 

the learning. Rothkopf initiated a series of studies on the effectiveness 

of questions placed before the text passages concerned, inserted into them 

or placed after them. Not unexpectedly it was found that, whereas 

introductory questions tend to lead the study to what would answer them 

specifically to the detriment of the study of other parts of the text, 

those placed after the text passage have a more general effect, stimulate 

more careful learning and lead to slower learning of later passages. The 

delaying effect seems to disappear gradually, maybe because better study 
skills have been acquired with the help of the questions. 

Research at the Open University 

'supports, but so far does not add to, the practice of inserting 
into texts higher-level (not rote recall) questions after the relevant 
teaching material. This practice was adopted at the Open University in 
1969 on grounds of common-sense, teaching experience and the 
distilled experience of practical work on programmed instruction'. 

(Macdonald-Ross 1979 p. 24) 

Anne Benko de Rotaeche uses Ausubel's and Rothkopf 's approaches as starting 
points for a study of the influence of teaching strategies applied to 
written material. 

Her investigation is based on the study of the effectiveness of three 
different versions of a course used by the Universidad Nacional Abierta of 
Venezuela. Although the student sample concerned is small, the study would 
seem to be of considerable interest. 

While the theme studied is highly relevant to distance education and the 
investigation painstaking and valuable, research of this kind is 
methodologically tricky. For this reason a comment by Dr. Rudolf Schuemer 
on the methods applied and possible and the author's reply to this are 
printed in an appendix. 

Our Institute, ZIFF, is grateful to the author for her permission to let 
us print her study (originally a Pennsylvania State thesis). 

B* Holmberg 
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ABSTRACT 

The purpose of this study was to determine whether the variables 
found to influence learning in laboratory settings were able to produce 
effective student achievement when utilized in Venezuelan distance education 
study material. Three versions of the same instructional content were 
employed. Forty-two volunteer college students were randomly assigned to 
three experimental groups. Group 1 received an instructionally designed 
text in which such aids as objectives, advance organisers, and exercises 
were interspersed along with the reading material. Group 2 received the 
same content with a different presentation; text materials and instructional 
aids werde separated into two booklets. Group 3 received the same test 
materials without any instructional aids. A posttest-only design was employed. 
Analyses of the results obtained on the posttest indicated that Group 1 
performed at a significantly higher level than Group 3. Organization, 
condensation, and content that meets the interests of an adult learner were 
the factors in the study materials found to be most effective for these 
students. No significant effects a to age, study habits, or educational 
levols y^re found. There was evidence that different versions stimulated 
different learning activities and that higher achievements were attained 
when the instructional aids were taken into account. It was also found 
that instructional design was able to stimulate a higher level of 
processing. Although all students obtained the lowest scores on items 
requiring deep processing. Groups 1 and 2 attained higher mean scores than 
Group 3; the deepest levels were attained by subjects who reported that 
they applied the study techniques recommended in the instructional material. 
This study has implications for the type of instructional materials to be 
used in Venezuelan distance education. 
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Chapter I 
INTRODUCTION 
Context of the Problem 

During the last twenty years, the learning imd relontion processes 
involved in the study of written materials have attracted a great dea] 
of research. There are probably two reasons for this: one practical 
and the other ^uOoretical . On the practical side, texts constitute a 
major medium of learning and are usually used in instructiona] settings 
as basic or supportive teaching materials. If wc add to this the 
increasing need of civilized man to keep informed and the \;ide demand 
for educational services at all levels, including the burgeoning 
development of educational innovations for adults such as continuing 
education, open universities, and distance education, it is easy to 
understand the interest in developing strategies for writing instruc- 
tional materials capable of influencing learning and retention. 

On the theoretical side, probably the strongest impulse came 
from the influential work of Ausubel on meaningful verbal learning 
(1968) and the notion of "mathemagenic behaviors'* introduced by 
Rothkopf (1965). In a period uhen many psychologists were still 
studying associative learning, Ausubel pointed out that typically 
learning in the classroom is qualitatively different from this, and 
that meaningful verbal learning deserved to be investigated. ilis 
emphasis on cognitive processes and structures showed the iriportancc 
of controlling the presentation of instructional materials and the 
influence of the knowledge students bring with thc»? to the Ica^'ning 
situation. *'If I had to reduce all educational p^wchology to jusl ciu- 
p/inciple, I vould say this: The most important single facto:* 
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influencing learning is what the learntr already knows. Asc rtain 
this and teach him accordingly" (Ausubel, 1968). Although Ausubel's 
ucvk drew attention to meaningful prose learning, it was Rothkopf, 
according to Faw and Waller (197A), who showed h(^w i]westi^\at:ions 
should be carried out in this field. He suggested that subjects, when 

jdying written materials, not only learn the specific content but 
also acquire sone general facilitative skiJl, namely "inspection 
behaviors" which he later called "mathemagenic behaviors" (tlothkopf, 
1966). Although critici^ed for its imprecision (Rickards, 3979), 
this notion served to "remind resecirchers that what the student does 
in the learning situation is an important key to how much he will 
retain" (Faw 6f Waller, 1976). Rothkopf created an experimental 
paradigm which allowed the assessment of a "direct instructive effect" 
and an "indirect effect," the so-called mathemagenic behavior 
(Rickards, 1979). Since 1965, research has taken on two orientations: 
one directed toward the influence of different vari ables such as the 
effects of question-answering on text retention, and the other directed 
toward the processes involved in learning from textual materials. 
Specifically, in the field of adjunct questions, research influenced 
by the behavior ist ic model, originally studied the effects of question 
frequency and position within prose. Later, influenced by the general 
trend of experiemntal psychology toward the cognitive model, investi- 
gators studied the effects of levels of questions on the depth of the 
cognitive processes (Rickards & Denner, 1978). As the review of the 
literature will show, learning is no longer seen as a "mere agriega- 
tioii of information" but as a "dynamic structure-imposing process. 




resulting in schema changes" (Anderson, 19/7). In tliis view, schemata 
are essentially organizations of information in holistic and hierar- 
chical structures. 

Today, a substantial body of knowledge is avaiJablc on the many 
variables that may affect learning from prose adjunct pre- and post- 
questions, advance organizers, text organi; -^tion, feedback, etc. 
Generally, the studies have been conducted in laboratory settings. 
The paradigms used assess the effectiveness of each teaching strategy 
in isolation, or of a cluster of several strategies, rather than their 
interrelation within a total learning situation (Harlley ^ Davies, 
1976). The maximum passage length reported for these studies was 
23,000 words in the research by Custafson and Toole (1970). 

Many authors (Faw St Waller, 1976; Rickards & Deniier, 1978; 
Rickards, 1979; Gagne, 1978) recommend that future investigations try 
to keep experimental conditions as close as possible to everyday 
practical situations. Some of them (for example, Rickards, 1979) also 
point out that new models are needed to answer relevant questions. In 
spite of their large number, most of the csearch^rs arc decision- 
oriented and a complete theory of instruction is otill lacking (Faw & 
Waller, 1976). 

Based on these conditions, the present study aimed Lo determine 
whether the deliberate use of a cluster of teaching strategies in 
written material could influence learning positively. The setting 
chosen for this project was not a laboratory, but a real teaching 
situation. Since instruction is mainly conveyed tiirough written 
materials in distance education, real studies in a distaiice ui.ivt'rsii> 
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were used as subjects to test their reactions to real instructional 
raaterial. The students selected had enrolled in a Venezuelan dis- 
tance university, the Universidad NacioPc^l Abierta (U.N.A.). This 
university offers professional degrees in five careers (professions). 
Its objectives are to develop human resources in national priority 
areas such as education, engineering, business administration, and 
tr.atheraaLics. U.N. A. attempts to cooperate with the higher education 
system of Venezuela to implement educational innovations and to meet 
a strong demand for education in the whole country. 

Distance education is a mode of education in which instruction is 
given without any contact between teachers and students. In its 
origins, distance education used correspondence courses as a means to 
convey instruction. Today, and specifically at U.N. A., distance 
education employs an integrated combination of means such as televi- 
sion, radio, audio cassettes, computers, and printed materials. Many 
countries throughout the world, including the U,S.S.R., France, 
Australia, Sweden, Japan, and Canada have developed distance procedure 
for higher education. More recently, new institutions have been 
created that combine distance education with the principles of open 
education. These institutions fall into two major groups. Some are 
more traditional: they offer distance courses, either on their own 
cr in cooperation with other universities (for example, Universidad 
Autonoma de Mexico, The German Institute of Distance Education at 
Tubingen University, and Tlie Pennsylvcjnia State University). Venezuel 
has had twelve years of experience in this type of arrangement at the 
Uni-vcrsidad Central de Venezuela and some other universities. The 




.•tilor irond is exemplified by new, self-sufficient ins r iCiit ions , 
•.':":ch develop their own courses and confer prof essioiial degrees (as is 
t::c ca>e of the Open University of Great Britain, Distance Education 
j:" luigcn in Germany^ the Universidad a Distancia of Spain, Everyinan^s 
Iniversity of Israel, and U.N. A. in Venezuela ) (Universidad Nacional 
ASierta, 1979). 

Distance education is a field of major concern in I.atin America 
since ic represents an attempt by several governments to meet an 
increasing demand for higher education. Examples are tlic Open 
Universities of Brazil, Columbia, and Costa Rica. U.N. A. is the 
largest developing project in the context of distance education in 
South /\merica. 

The material studied in this investigation constituted a complete 
"unit" of one of the courses which forms part of the curriculum in 
the field of Education, At U.N. A., a course is divided into "modules" 
which, in turn, are subdivided into "units." Modules are tlie princi- 
pal parts into which the course content is divided; they constitute 
organized bodies of content and represent the general objectives of 
the course. Units are the elements of a module. Since this study was 
focused on a unit, its principal parts will be described. A unit h?5 
the following structure: (a) an introductory section, which includes 
the staterr.ent of one or sevc^.ral objectives, general recomnendai ions to 
the student, one or several advance organizers, and an operational 
pretest; (b) the main section of the unit, which contains r.lie inslruv:- 
rional information, wliich is an organized body of knowledge including 
definitions, explanations, examples, pre- and/or postquesl i ons , an^l 



corresponding feedback; (c) a post test, followed by the correct 
answers and recommendations to the student ("prescriptions"). As can 
be seen, the elaboration of a U,N,A. instructional unit follows an 
"instructicnal design/* In other words, it attempts to incorporate 
Che accumulated knowledge derived from teaching models, learning 
theories, and experience in distance education. 

Statement of the Problem 
The purpose of this study was to determine v;hether: (a) variables 
found to influence learning in laboratory settings are able to produce 
effective student achievement in a common teaching unit in Venezuelan 
distance education, (b) such achievement is related to age, educational 
background, or study habits, and (c) students are able to use deeper 
cognitive processes more efficiently as a result of the instructional 
manipulations. 

Expectations 
The following expectations were formulated: 

1, Three versions of the same written instructional material 
will result in differential student achievement, 

2, Three versions of the same written instructional material 
will result in differential student achjcvenient depending 
on age, educational background, and study behavior. 

3, Three versions of the same writter instructional material 
will result in differential levels of processing. 
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RationalG 

Studies in laboratory settings liave generally demonstrated the 
effectiveness of several prcinstructional strategies (Hartle> & 
Davies, 1976) in student learning. The passages used in these studies 
were generally short (Faw & Waller, 1976). It seems plausible chat 
these strategies, if deliberately employed as a group, would influence 
learning positively. It was also to be expected that, as so"ie authors 
found (Annbruster , 1976), the results obtained using a common instruc- 
tional unit in artificial settings would be obtained in natural 
settings. 

In addition, since cognitive psychology presumes that the cogni- 
tive structure is a network of schemata and the interpretation r>f 
particular situations consists of matching elements in the situations 
with generic characterizations in the schematic knowledge structure 
(Anderson, Spiro & Anderson, 1978), it could be expected that students 
having some cultural background — namely some experience in higher 
education, recenu study in secondary school, or good study habits — 
would be more likely to apprehend and learn the concepts of the unit 
than other kinds of students. 

Finally, since Greeno stated that a text which is able to show 
students the relationship among the parts of a structure is preferable 
for teaching problem-solving strategies (DiVesta, 1974), since 
Klausmeier et. al. demonstrated that the use of definitions plus 
examples enhances learning (Armbruster, 1976), and since several 
authors contend that higher-level questions promote a deeper processing 
of the instructional material than lower-level questions (Watts 6* 
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Anderson, 1979, for example), it would seem that an instructional 
unit in which a deliberate use was made of such strategies would 
result m positive transfer to new situations and in deeper cognitive 
processing than instructional material that does not make use of 
these strategies* 

Operational Definition of the Variables 
The principal variables studied in this investigation are 
operationally defined below. 



In order to test null hypotheses, three versions of the sair.c 
instructional information were created. The independent variable 
was the version of the instructional material that v xs studied by 
the students. 



The dependent variable, achievement by the students, was deter- 
mined by having them respond to a posttest. With regard to the third 
hypothesis, achievement was studied at three levels. That is to say, 
the posttest was composed of three levels of questions in order to 
determine three levels of processing, as follows; (a) Level A — 
students would demonstrate mastery of information by being able to 
recall and make associations between ideas; (b) Level B — students 
would demonstrate that they were able to abstract important ideas of 
joncepts, to establish relationships among concepts, to translate 
communications into other languages, to apply correctly the right 
procedures to solve a new proble'", to analyze, generalize, and 



Independent Variable 




Dependent Variable 
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classify properly; and (c) Level C — students would demonstrate the 
aVility to combine different elements in order to construct a new 
j;tructure which was not present before, and would demonstrate their 
ability to write a unique synthesis of ideas. 

Moderator Variables 
Hypothesis 2 took into account moderator variables which could 
affect the results. The moderator variables considered in this study 
were the following: (a) age (defined as 20 to 30 years old, and 
30 years of age and above); (b) educational background (defined as 
completion of high school, technical studies, higher education); and 
(c) study habits (defined as activities engaged in during the study 
of the instructional materials, such as rereading of main parts, taking 
notes, making abstracts, underlining important parts, following the 
sequence, completing exercises, answering every question, comparing 
their own answers with those of the instructional material, reading of 
additional bibliography, seeking counseling). 

Significance of the Study 
It was hoped that this study would help to identify the extent to 
which a deliberate manipulation of the presentation of information in 
an instructional unit would influence the achievement of distance 
education students. Such information could assist teachers anj admin- 
istrators of distance education in reaching decisions about what kinds 
of material can best be used to meet the needs of adult students. 
Therefore, it could be a step toward improving the planning of distance 
education in Venezuela and other countries. 
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Chapter 11 
REVIEW OF THE LITERATURE 
In this chapter, the principal experimental findings related to 
each one of the teaching strategies employed at U.N»A. in its instruc- 
tional material, including the unit under study, will be summarized. 
The results of the existing interrelations, and the theoretical 
explancJtions postulated will also be reviewed. 

Introductory Strategies 
Because they are considered to be powerful determinants of 
motivation, comprehension and retention, introductory strategies are 
usually chosen and prepared carefully. Each one of their hypothetical 
fincnions — to familiarize the student with the content, to provide a 
framevv'ork in which subsequent learning can be subsumed, to relate what 
is to be learned to what has already been learned, and to enhance 
discrimination — has been studied in exf>erimental contexts. However, 
as Hartley and Davies (1976) point out, few attempts have been made to 
compare the practical effectiveness of each strategy. 

Pretests 

Since no pretest was used in this study, the effects of pretests 
upon learning will not be reviewed. 

Object iyes 

Objectives are deliberately designed to improve the quality and 
effectiveness of teaching. Their purpose is to inform students of 
what they are expected to do after studying some material. As 
Gagne (1965) argues, objectives facilitate study since they provide a 
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clear goal that helps students organize learning activities, reduce 
time wasted on ir relevancies , and offer a basis on which they can 
objectively assess their own progress. For these reasons, much care 
must be taken to state objectives as unambiguously as possible, and a 
whole technology of writing cbjectives has arisen. So.ae extreme posi- 
tions urge that, in writing objectives, intangible goals such as 
"understanding*' and ''becoming aware of" should be avoided and replaced 
by overt goals which can be. reliably observed and measured. It has 
been said (Mager, 1962) that an objective should specify the kind of 
behavior that the student is expected to perform, the conditions under 
which such behavior should appear and the criteria which will be used 
to confirm whether the level of performance is acceptable. 

Considerable controversy has emerged as a result of this movement 
Many authors and teachers claim that excessive specificity and emphasis 
on detail, which frequently occur in long lists of specific objectives, 
may draw attention to irrelevant tasks, at the expense of more genera] 
humanistic and intellectual goals. In fact, a course is usually con- 
ceived in terms of its main goals which are expressed as general 
objectives* In order to plan instruction, these general objectives 
are broken down into specific ones that are intended to direct 
learning progressively toward mastery of the general objectives. 
Critics contend that the efforts of the instructional designer to 
identify those steps and tasks conducive to the final behavior are 
frequently ineffective because fundamental aspects linked to the main 
gOt^s of a course, such as attitude formation and the establishment of 
relationships among parts, are forgotten or obscured by the details. 
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The experimental findings related to the effectiveness of 
objectives deserve comprehensive review. For example, when studies 
that did not show positive effects (Jenkins & Deno, 1971) are 
analyzed, their negative results m^y be explained by several uncon- 
trolled factors, such as the use of a nondiscriminatory test, con- 
siderable prior knowledge on the part of the subjects in the 
experimental and control groups, or interference resulting from 
definitions of the objectives implicit in the organization of the 
material (Faw & Waller, 1976). 

Other studies showed a positive effect of objective-related 
learning (Rothkopf, 1966; Duchastel & Brown, 197^; Lavson, 197^; 
Gagne & Rothkopf, 1975), often at the expense of incidental learning 
(Prase & Krietzberg, 1975; Rothkopf & Billington, 197^; Gagne & 
Rothkopf, 1975). Duell (197^) concluded that objectives stimulate 
the learning of only such material as is considered unimportant by the 
subjects. Student expectations of what will be on a test are influ- 
enced by the type of material studied, prioi <ixperience with tests, 
and the type of course presented. In order to be useful and not merely 
to overlap with text organization, behavioral objectives should include 
major concepts which are not already indicated in the text. These 
would provide guides which differ from the orientation provided in the 
written passage. 

The conclusions of the investigations related to the influence of 
objectives on learning may be summarized in the following manner: (a) 
specific objectives enhance intentional learning rather than inciden- 
tal learning; (b) specific objectives exert a greater influence on 



intoncional learning when they are distributed throughouf. each section 

the text, rather than presented all together; (c) general objectives 
facilitate incidentdl learning. Apparently, objectives nelp students 
organize ir.aterial into hierarchical structures and derive implications 
c\nd applications from the instructional material. 

In an attempt to explain how objectives operate upon the cognitive 
structure, Gagne and Britton (1977) suggested that objectives act upon 
attention and rehearsal. Objectives placed before a reading passage 
alter attending and encoding strategies. OL^.-^ctives provided before 
review influence rehearsal. Using the ACT ossociationist model of 
memory proposed by Anderson, Gagne hypothesized that using poslobjec- 
tives as a guide to construct self-questions may strengthen the path- 
ways used. Long-term recall should improve when self-questions match 
post test questions . 

Advance Organizers 
According to Ausubel (1960), an advance organizer is an outline 
of the novel ideas that will follow, presented at a higher level of 
generality and abstraction than within the text itself. The introduc-- 
tion of the unit used in this study constituted an advance organii'^cr. 
The strategy of presenting an organizer prior to the instructional 
material is based upon Ausubel *s assumption that meaningful acquisi- 
tion and retention of ideas are enhanced if some general or inclu- 
sive principle, to which the ideas can be related or under which they 
can be subsumed, is offered before they are described specifically 
and in detail. In fact, Ausubel postulates that cognitive structure 
IS hierarchically organized in terms in "highly organized concepts 
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under which are subsumed less inclusive subconcepts and informational 
data/' The av liability of appropriate and stable subsumers should 
enhance the incorporability of new materials. Hence, before intro- 
ducing unfamiliar academic material, a suitable organizer is provided, 
depending on the subsuming concept. Ausubel states that, as teaching 
strategies, integrative organizers are more appropriate to abstract 
content than Informational data. One of the main characteristics of 
an advance organizer as conceived by this author is that it is 
"relative/' i.e., it depends on the learner and the subject matter. 
A passage that serves as an advance organizer for one learner may not 
be needed for another learner. The ideal situation would be to 
"ascertain what the more common preconceptions of learners are by 
means of appropriate pretests and then to match suitably tailored 
organizers with pupils exhibiting corresponding preconceptions*' 
(Ausubel, 1968), Wien a learner lacks prerequisite subsumers, an 
^'expository organizer" is recommended to provide an explicit background 
under which to subsume the specific concepts which follow. When the 
learner has the prerequisite subsumer, a "comparative organizer" seems 
to be useful to emphasize the relation between the existing subsumer 
and the information which follows it. However, as Mayer (1979) pointed 
out, "further research is needed to determine the best analogies, 
images, examples, etc., that serve as effective advance organizers for 
individual subject matters and learners." 

Recently, considerable controversy was aroused by a review of the 
literature on advance organizers by Barnes and Clawson (1975). After 
reviewing 32 studies, of which two failed to produce significant 
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results, they concluded thc»t "ndvnncc organizers, as presently con- 
structed, do not facilitate learning." In his defense of advance 
organizers, Ausubel (1978^^ outlined sevora] nicth(Hl(>h)j;ic .il miscoticop- 
tions that explain these adverse conclusions, lie cited sever«ii 
studies of conceptual adjunct prequestions (F<ickards 6 DiVesta, }97^; 
Rickards & Hatcher, 1975) that **indicnLc that once the roLc JearninR 
methodological bias of Rothkopf and Prase (requiring- verbal ini recall 
of single text phrases) is discarded in favor of substantive learning 
of entire paragraphs, . . . superordinate concepts in the adjunct 
questions facilitate the lc;irnin}» of snhord i nat o loxliiai material in 
much the same way as advanec orp,anix.ers do.*' lie also roTerred to a 
study by Lawton and Wanska (L977) that denionslratc^d an ateeJeralin^ 
and faci 1 iLatinp, effect of advance orj',ani x.crs in 6- and lO-vear-old 
children 

Mayer (1979) demonstrated the effectiveness of advance organizers 
in facilitating far transfer problems and recall, compensating for 
text organization, assisting discovery learning, facilitating the 
acquisition of higher-order rules, anr! linear reasoning. He suj^gesled 
that high-ability subjects do not need advance organizers, and that 
low-abi]it> subjects are more likely to benefit from them since they 
offer a framework that voiOd not otherwise be available. 

Mayer proposed an assimilation encoding theory to explain the 
effects of advance organizers. It is a three-stage model. In the 
first stage, the information is received by the working memory from 
the outside world. The second stage corresponds to what is anchored 
in long-term memory; it refers to available knowledge. The third 
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stage consLitutes the trans»'er of the nncliorcd luu)wJc(l|;c from long- 
term memory to working memory, and the integration of that knowledge 
with incoming information during learning. Each stage includes (a) an 
encoding process which is an active inLogration of new information 
with existing knowledge; and (b) a learning outcome which varies as 
much in amount of information as in breadth. This theory postulates 
that giving the organizer before learning may resul i • in the activation 
0 availnhJe |)rior knowledge and encoding processes wliich, in turn, 
results in a "broader learning outcome." 

Indivi dual Differences and Advance Org an izers 

Smith and Hesse (1969) observed that advance organizers may have 
different effects upon readers of high and low ability. They found 
that organizers have negative effects on comprehension with good 
readers, but positive effects for poor readers. Smith and Hesse con- 
cluded that good re lers have their own organizational styles which 
may conflict with the text organization, while poor readers do not 
have organizational strategies and, consequently, benefit from advance 
organizers. 

Instructions 

Because it is assumed that student activity aids learning, some 
guidelines are generally given prior to presenting written iiistruc- 
tional materials. Any of the following recommendations may be given: 
rereading, underlining, outlining, note-taking, diagraming, and 
responding to inserted questions. 
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The empirical evidence as to the teaching effects of these 
activities xs still confused, Ncf significant differences were found 
among the respective values of rereading, underlining, outlining, 
and precis writing, in a study by Arnold (19A2), With regard to note- 
taking, several studies showed positive results (Fisher & Harris, 1973; 
DiVesta & Gray, 1972, 1983; Aiken, Thomas & Shennen, 1975), but other 
studies showed detrimental effects (Peters, 1972), 

Trying to explain these confusing results, Faw and Waller conclude 
that, as suggested by Weener (197A) and Peters (1972), a good memory 
may be needed to take advantage of notes. Note-taking may have two 
beneficial effects on learning. First, it may exert an encoding or 
organizing function since it involves processing of information. 
Second, a review of notes at some later time is likely to be helpful. 
Faw and Waller (1976) noted that the experiments in which the experi- 
mental subjects showed a better performance than the control subjects 
were those in which the rate of presentational material was slow. 
They hypothesized that note-taking may interfere with attention and 
the on-going reception of material when presentation rates are fast. 
In distance education, taking notes may be compared to the experimental 
situation in which the rate of presentation of the material is very 
slow, since students learn at their own pace. Nevertheless, what 
should be taken into account in this type of learning is that there 
are probably variations in the encoding mechanisms usec^ by students of 
distance education; some may record notes verbatim, while others 
elaborate organizational schema. As DiVesta and Gray (1972) assert. 
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•*JHVC5» t igat ions on such variations should provide considerable infor- 
mation on the cognitive processes." 

Instructional Strnte^>ies 
Research on the variables that may have significance for learning 
and retention of prose is relatively recent since one of the major 
difficulties has been how to operationalize the structure of the 
r.aterial for its use in experiments (Duchastel, 1879). The most 
important variables studied have been the following: organization of 
the passage and the relative importance of information within that 
context, the use of definitions, principles, and examples, the extent 
to which a passage stimulates imagery, the function of inserted ques- 
tions of different levels of complexity, and the amount of redundancy. 

Organization of the Instructional Material 
Studies of the effects of prose organization on the learning 
process and the amount learned are relatively recent. The main reasons 
for this are, partly, the belief shared by many investigators that 
learning processes could be studied most effectively with less complex, 
more easily controlled stimuli (e.g., lists of nonsense syllables oi 
words) and, partly, the lack, until recently, of reliable procedures 
for text analysis. Research has now demonstrated that the results 
obtained from earlier studies with syllables, words, and short sen- 
tences cannot be extrapolated to learning from prose, and that the 
respective results are frequently contradictory (Mayer, 1977). 
Probably the first attempt tL relate the organizational characteristics 
of a text to cogntive processes was that of Dawes (1966). The it^ain 
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devalopmenCs in that field appeared around 1972, when psychologists, 
like Crothers, Fredericksen, Kintsch, and Meyer, constructed pro- 
cedures for prose analysis by adapting linguistic procedures, like 
those of Fillmore, and procedures from the area of artificial intel- 
ligence, lik those of Simmons, Simmons and Slocum, and Schank (Meyer, 
1975). 

Most of the texts used at U.N. A. are organized around key con- 
cepts that unify or subsume the component ideas. Cognitive psychology 
assumes that broadly inclusive categories help learners to organize 
parts of knowledge into thoir cognitive structures . Specifically, in 
the framework of schema theory, cognitive structure is a network of 
"schema'ca'* which are the equivalent of concepts, i.e., "the represen- 
tation of a category, a general knowledge *,hat incorporates that 
which i.s common to a large number of situations or things" (Anderson, 
Spiro & Anderson, 1978). So, to "interpret a particular situation in 
terms of a schema, is to match the elements in the situation with the 
generic characterizations in the schematic knowledge structure . . . 
Another way to express this is to say that schemata contain slots or 
placeholders that can be instantiated with certain particular cases" 
(Anderson, Spiro C Anderson, 1978). Following Kintsch (1977), it may 
be said that the organization of a text is usually guided by a 
''schema. " In this content, "text schemata are general principles 
observed by any culture in constructing particular types of texts." 
In Occidental culture, texts have a prepositional base, which 
represents the essence of the text; when a pjssage is long, the text 
material has to be presented as a sequence or hierarchy of subunits. 
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This organization is called the macrostructure of the text. Each 
subunit is a limited set of propositions, subordinated to a subject 
or "label" for that unit, called the "macrostructure proposition." 
The analysis of a text must then specify how the division into subunits 
is achieved and how the macroproposit ion for each unit is derived from 
the text. Meyer (1977), for example, described a procedure of division 
which resulted in a tree structure to represent the relationship among 
the ideas in a text. In order to test the reliability of her proce- 
dure, Meyer had two independent judges select the most important ideas 
in a passage and relate all the component ideas they believed were 
logically subsumed under the general statement. She obtained an inter- 
judge agreement of 91.5 percent for her procedure which is still 
widely used among experimenters. 

Studies have also been conducted in order to test the relationship 
between text organization and cognitive processes. One area of inter- 
est has been, fo/ example, the relationship between the importance of 
ideas in a text and recall. The results point to the fact that impor- 
tant ideas are better recalled. In order to support the ACT model of 
memory that she adopts, Gagne (1978) compared the studies of Johnson 
(1970), Meyer (1977), and Kintsch, ec al. (1975). Johnson had college 
students rate idea units on their importance and found that ideas 
rated as more important were freely recalled with greater probability 
than less important ideas, on both immediate and long-term tests. 
However, Meyer and Kintsch et al. found an interaction between impor- 
tance of the ideas and retention interval, in the sense that increased 
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retention of superordinate elements was observed immediately after 
learning and to an even greater extent seven days later. 

Although Meyer assumed that these results provided support for 
the subsumption theory, Gagne considered an alternative interpretation 
possible in the light of the ACT model of Anderson. This model 
hypothesizes that: (a) Information is represented in memory by nodes. 
Important information is represented by a larger number of intercon- 
nected nodes, (b) Tliere is a greater probability of recalling impor- 
tant information because a greater number of retrieval cues are 
attached to it. (c) The greater number of retrieval cues will result 
in more frequent rehearsal. (d) The result of this frequent rehearsal 
is that important ideas will be recalled with greater porbability 
than unimportant ideas at long term. Gagne explained the failure of 
Johnson to obtain a progressive effect by the fact that his subjects 
did not know that they were going to retested, and therefore they 
probably did not rehearse as much as the subjects in the other two 
experiments. 

The field is full of conclusions of this kind in which compari- 
sons are difficult because of the different variables taken into 
account. The interpretations are also controversial, depending on 
the theory adopted. Several attempts have been made to clarify why 
important elements are better recalled. As Pichert (1979) pc its out, 
this may be related to different strategies operating at the moment 
of encoding or different strategies operating at retrieval. Among 
encoding possibilities, two may be mentioned: (a) after identifying 
important elements, the learner directs a greater amount of attention 
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and cognitive processing to them; or (b) the learner uses text 
elements to fill the slots in an existing knowledge net. Material 
will be better recalled if it fills the available slots. At retrieval, 
several explanations are also possible: (a) memory search proceeds 
from generic knowledge to the particular information stored when the 
text was read; (b) inferential reconstruction takes place. When 
failing to recall a particular text element, a subject might try to 
"reconstruct it on the basis of items which usually fill the blank 
slots in the operative knowledge.** Pichert's explanation is consis- 
tent with Meyer's position. Their views are important in deciding on 
teaching strategies, since they empliasize the importance of what the 
student already knows. This, in turn, takes us back to Ausubel*s 
position that the design of instructional materials should start from 
an evaluation of what the learner brings to thr learning situation. 

Attempts have been made in other contexts to demonstrate the 
hypothesis that r'^ading comprehension, aiid hence learning, involves 
filling in specific textual information to abstract schemata. For 
example, Pearson, Hansen, and Gordon (1979), reasoning that a reader 
who has a better developed schema for a particular topic should under- 
stand and remember more than one with a weaker schema, applied that 
assumption practically in a natural environment, using second-grade 
pupils. Children who demonstrated greater prior knowledge showed more 
comprehension of explicit ideas and of ideas requiring an integration 
of prior knowledge and text information. 

Spiro (1979), working on the selectivity of memory, argued that 
when reading, we superficially process information, especially when 
a schema related to the topic already exists. When recall is required, 
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we just reconstruct on the basis of elements that are topic-related. 
Meyer (1977) discussed this position as follows: '*In non-experimental 
sutations Sprio would predict that recall of groups of people would 
not relate to the content structure of a discourse, but instead 
uniquely to the individuals, as directed by their varying interests 
and existing knowledge." Meyer accepted this position, but she stated 
that in typical school situations the learner generally must ascer- 
tain the writer's or speaker's message. The reader must have a 
strategy for dealing with the message and '*the most efficienc strategy 
is to utilize the schema or top-level structure of the writer to 
organize the information for storage in memory." In order to demon- 
strate this assertion, she cited the Meyer and Freedle study (1979) 
in which they showed that, when the reader's schemata do not corres- 
pond to or are even contrary to those of the author, recall will not 
necessarily relate to the content structure of the passage. Never- 
theless, Anderson et al. (1977) criticized her position in a manner 
that may be considered somewhat misleading. Specifically, they 
wrote: 

. . . there is not reason to suppose that it is only 
when attempting to understand ambiguous passages . . . 
that subjects bring to bear extra-linguistic knowledge. 
Indeed, there is every reason to believe that language 
comprehension always involves using one's knowledge of 
the world. 

This criticism may be misleading since Meyer was arguing about the 
best strategy for a studenu to meet the author's — and hence the 
teacher's — criteria, and she described a situation in which 
this is in fact very difficult because the respective schema 
are contradictory. Later Anderson et al. (197?), referring to 
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Wycr'iJ work (1977), underlined the fact tl»at ''the implications of new 
information will be resisted if its acceptance would require a major 
cognitive reorganization,'* which looks like a plausible explanation 
for the same fact noted by Meyer. It is probable, they hypothesized, 
that people are able to keep materials segregated from logically 
incompatible beliefs. The implications of such subtle mismatches are 
very important for the evaluation of a learning process and this is 
probably why so many taxonomies refer to high-level objectives 
(application, analysis , synthesis) . 

The effect of the style used for written materials upon achieve- 
ment and retention has also been studied. DiVesta (195^) compared 
three styles (expository, popular, and study guide) in a study applied 
to correspondence course students. He did not find differences in 
their relative effectiveness. Nevertheless, and since the experiment 
is similar to the present one considering the sample and the text 
used, it is worthwhile noting that "the administrative procedures 
required in conducting the experiment with correspondence course 
populations were found to be too ponderous for practical purposes." 

The effect of signals or cues in prose has also been studied. 
For example, Meyer (1977), in a study conducted in 1975, used 
signaling devices such as statements like: "There are tv:o approaches. 

One is , and the second is ." She found a tendency 

for such devices to improve recall, but the effect was not statist - 
cally significant. She hypothesized that the smallness of th'^. effect 
was probably due to the fact that "signaling was aimed primarily at 
information high in the context structure." As can be seen, there 



exists an interrelation between text organization and cues; cues are 
less effective when the text is hierarchically organized. On the 
other hand, Glynn and DiVesta (1979) showed that a single sot of 
cues — instructional or typographical — increased recall of cue- 
related propositions, but that two sets of conflicting cues reduced 
recall. Confirming previously mentioned findings, when no cues were 
at hand, recall was best for propositions that were conside.red impor- 
tant by the learners. An important point for text organization is 
that recall was best when related information was grouped topically. 
The conclusion of this research was that cueing systems guide prose 
processing activities. Nevertheless, it is important to relate these 
results to the observation by Meyer (1977) that "substantial amounts 
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of signaling robbed information from non-signaled information low in 
the content structure, and subsequently reduced its recall." 

The teaching methods used throughout a passage may be based on 
discovery or on expository techniques. In 1972, Creeno stated that 
discovery tends to be an eff^^ctive procedure when the student has 
relevant prior knowledge. Expository techniques tend to be more 
effective with students vho lack previous information. The first 
technique is hypothesized to function as a comparative organizer since 
it is assumed that it hel{,s to incorporate new material into the cog- 
nit ive structure; the second is supposed to establish subsuniers under 
which new material can be organized, such as an expository advance 
organizer (DiVesta, 197/.). In this concext it is worthwhile remem- 
bering that, as Greeno stated, a text which is able to show studencs 
the relationship among the parts of a structure is recommendable for 
teaching problem-solving strategies (DiVesta, 197A). 
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Use of Definitions and Examples 
In an overview of studies of concept and principle learning, 
Armbruster (1976) distinguished between laboratory experiments and 
experiments conducted under conditions more closely related to educa- 
tional situations. So many differences between the laboratory and 
school learning situations have been listed that, following Armbruster, 
they will be considered irrelevant to the practical purposes of this 
paper. 

With respect to experiments in more natural settings, Armbruster 
pointed out that the results are contradictory and confusing but could 
have promising interpretations in the light of schema theory. 

As to procedures in concept-teaching, specifically using examples 
only (without definitions), a study by Swanson (cited by Armbruster) 
demonstrated that subjects given both positive and negative examples 
learned the concepts better (with less over- or undergeneralization) 
than subjects given either only positive, or only negative examples. 

Research on the use of both examples and definitions yielded 
important results for the elaboration of teaching materials . Using 
the words of Klausmeier et al. (reported by Armbruster): (a) . . 
the number and type of instances presented are less important when a 
concept definition is provided than when instances alone are used to 
teach a concept;" (b) "examples used in conjunction with definitions 
enhance learning;" (c) "a definition plus examples is superior to 
either condition by itself;" (d) "the relative effectiveness of 
examples or examples with rules depends on the nature of the expected 
performance." Coincidentally , in an experiment condurfi.^c by Guthrie 



(cited by Armbruster) on the learning of rules, the 'MUile plus 
Example" group was superior to all other groups in speed of learning. 

In relation to the use of definitions only, a study by Anderson, 
Kulhavy, and Andre (1972) demonstrated that a high level of learning 
is possible with definitions alone. But, as Klausmeier et al. 
observed, "research has not yet provided answers to such educationally 
importanc questions as: When introducing a new concept, how much of 
instruction should rely on verbally presented descriptions and defini- 
tions, and how much on presentation of concrete exemplars?" Armbruster 
claimed that schema theory may have the answer, '^e learning of a con- 
cept may be considered as the acquisition of an appropriate schema 
in memory. An "appropriate schema" is a network of inter relai:ions 
bearing the attributes or dimensions of a concept; it bears its own 
limits, neither too specialized nor too generalized. This definition 
leads us to a better comprehension of the role of examples in instruc- 
tional material. Adequate examples would be those that showed the 
nature and bounds of the variables which constitute a concept. The 
superiority of learning from definitions and rules can be explained by 
their providing "information in a structured form most closely 
resembling the structure of the schema which will be required for its 
interpretation . . . . It . . . maximizes the likelihood that the 
interpretation will be appropriate and minimizes the processing 
required" (Rumelhart & Ortony, 1977). Thus, the requirements for 
effective examples and definitions are very rigorous; they must com- 
municate an "adequate" schema. What constitutes an adequate schema 
is probably highly dependent upon the individual characteristics of the 
learner since it should fit the learner's personal cognitive structure. 
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When we turn to a consideration of the retrieval process, 
the research results become more equivocal. Relative effective- 
ness of methods is determined by performance on criterion tests. 
But according to schema theory, supported by the encoding 
specificity research, the effectiveness of a retrieval cue is 
a function of the ability of the cue to activate the schema 
formed on the occasion of the original input. Therefore, per- 
formance on a criterion test may not be a measure of the 
adequacy of schema formation at all, but merely of the similarity 
between the retrieval cue (test item) and the original input. In 
the absence of explicit information on the nature of th? criterion 
task and its relation to the learning task in most of the reviewed 
studies, it is not certain what the dependent variables measure. 
Unless the criterion task can be shown to measure the content of 
the supposedly encoded concept schema, the measure obtained is 
not an index of concept learning. (Rumelharl & Ortony, 1977) 

Tliis quotation illustrates very clearly the criteria used in the for- 
mulation of many of tht questions in the posttest of the present study 
where verbatim cues were avoided in favor of ''translation," "compar- 
ison," or "application" responses. 

Concerning the learning of responses, Armbruster (1976) presented 
a procedure for principle analysis following as closely as possible 
rhe structure of the schema, and attempted to test the hypothesis 
that a principle presented in conjunction with a domain statement (such 

Ps: "These principles apply to and not to ,") 

would produce a higher amount of learning than the presentation of a 
principle or example alone, or of a principle in conjunction with an 
example. Her experiment failed to confirm this hypothesis, but the 
analytical procedure might be a useful tool for educators and 
researchers. This procedure highlights what to teach and how to 
evaluate the effectiveness of teachinj; strategies. For research pur- 
poses, it shows a way of operationalizing the dependent and independent 
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Redundancy 

Another hypothesis that has received experimental attention has 
been Che possibility that the repetition of an idea influences the 
probability of its recall. Kintsch et al. (1975) found that the 
number of repetitions of word concepts was positively related to the 
probability of their immediate recall. Consistent with the Glynn and 
DiVesta study mentioned previously, \fliite and Gagne (1976) reported 
that two sentences having a linking concept were recalled better than 
two unrelated sentences. Another way of manipulating redundancy is 
the use of questions interspersed throughout the passage. Its effec- 
tiveness has been demonstrated repeatedly (Rothkopf 6f Kisbicos, 1967; 
Rothkopf, 1966; etc.). The differential effects of different kinds of 
inserted questions will be examined in a later section of this chapter. 

The ACT theory explains the influential effects of repetition by 
the use of old links and the addition of new links to the proposit ional 
network. The propos itional network is a set of nodes connected by 
links. The nodes represent ideas and the links represent relationships 
between those ideas. "The occurrence of occasions for practice of 
responses through reading should improve long-term retention either by 
strengthening links or, if the practice context is somewhat different 
from the initial learning context, by encouraging the formation of new 
links to the to-be-remembered nodes" (Gagne, 1978). 

Imagery 

Experimental research has demonstrated that imageable material 
(concrete words versus abstract words; picture pairs versus corres- 
ponding words pairs) is better recalled that nonimageable material. 
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iMivio (1971) •^interpreted these results as indicating that there are 
two independent systems of long-term storage, one visual and one 
\-erbal" (Gagre, 1978). The assumption of two storage systems is 
!)ardly discussed by other investigators, hov/ever. Anderson, t'or 
example, postulated that the facilitative effects of imagery occur 
because prior experience with pictorial stimuli is "richer than is 
experience with the symbolic counterpart of a pictorial stimulus" 
(Gagne, 1978). 

Among the few systematic studies of the interactive effects of 
pictures and text on learning from prose, are the findings of Stromnes 
and Nyman (197A) that immediate and long-term recall (one year) are 
enhanced by the combined use of both types of informr tion when com- 
pared to the recall of a text without pictures. "Nevertheiesc , 
pictures and diagrams to accompany printed matc^rials enhance compre- 
hension, if they are necessary and intrinsically related to the reading 
passage" (Bransford & Johnson, 1972). It is supposed that pictures 
exert a positive effect upon reading comprehension and learning when 
they show relationships, provide a basis for organizing information, 
and "present abstract concepts and principles at concrete levels of 
thought" (Singer & Rhodes, 1976). 

Interspersed Questions 
Since Rothkopf's work, the area of adjunct questions is probably 
the most developed in the field of learning from prose. As Rickards 
(1979) pointed out, the research turned from a "variables orientation" 
to a "processes orientation." In the earlier approach, Rothkopf 
(1966), for example, demonstrated Lhe effect of adjunct pre- and 
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poi?tquoscions on learning (variable orientation). The question — 
tirst raised by Frase (1967) — of whether the facilitative effect was 
attriluitvible to a ^'forward*' influence or a '^backward" review infJuence 
(process orientation) remained open for further study. Inserted ques- 
tions may vary in level of complexity; they may be verbatim or 
meaningful. These two broad calcigories may be studied according to 
their position (pre- and postquestions) , type, and frequency. 

Verbatim Inserted Questions 

Rothkopf (1966) studied the effects of factual (verbatim) 
questions inserted immediately before or immediately ai^-er related text 
segments, and measured their effects on incidental and intentional 
learning. Experimental groups surpassed control groups in intentional 
learning. With reference to incidental learning, questions-after were 
more effective than questions-bef ore; nevertheless, when compared to a 
control group instructed to read carefully, there were no significant 
differences. In further research, it was found that prequestions 
generally reduced the amount of incidental learning of the experimen- 
tal group relative to that of a no-question control group. 

Higher-Order Inserted Questions 

The research on this kind of question is of major significance in 
educational contexts. The effects of questions of different levels 
were first studied by Frase (1969). 

Higher-Order Prequestions . Frase (1969) compared the effects of 
factual questions with those of inferential questions, the latter aimed 
at the infere.ice of relations among different sentences in a passage. 



Although the inferential group did not demonstrate a clear superiority 
at drawing new inferences, it retained more information than did the 
group given verbatim prequestions. It is presumed that inferential 
prequestions lead to increased attention being given to a greater por- 
tion of information than do verbatim prequestions (Rickards & Denner, 
1978), 

Higher-Order Postquestions . Watts and Anderson (1971) found that 
postques tions that required the application of concepts or principles 
resulted in positive transfer in a new application situation. They 
interpreted this to mean that higher-level postquestions promote a 
deeper processing of the instructional material than low-level 
postquestions. 

Comparing the effects of rote-learning postquestions and rote- 
learning of ideas with postquestions of a "meaningful" kind, Rickards 
and DiVesta (1974) found that: (a) meaningful learning postquestions 
were more effective than verbatim postquestions; (b) meaningful 
learning postquestions were more effective when placed more frequently 
in the text; (c) all question types produced higher intentional 
learning than no questions at all; (d) meaningful learning postques- 
tions also produced a greater amount of incidental learning than no 
questions; (e) on a free-recall test, the meaningful learning post- 
question group presented better organized ideas than the other groups. 
It was concluded that such meaningful postquestions required the 
learners to relate subordinate passage content to general ideas in the 
text. 
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It is worth citing one comment with respect to the frequency and 
position of postquestions, "placing meaningful learning questions 
relatively far apart may . . . overtax . . . the subject's processing 
capacity, that is, produce excessive cognitive strain (Bruner et al., 
1956), thereby eliminating their advantage over other types of ques- 
tions*' (Rickards & DiVesta, 1974). "Verbatim postquestions , on the 
other hand, produce very little cognitive strain whether they were 
placed close together or far apart, since the processing behaviors 
associated with these question types were less extensive (Rickards 6 
Dcnner, 1978). 

Higher-Order Pre- and Postquestions Compared . A consistent 
position effect for conceptual questions has not been found. For 
each study in which the position and level of the adjunct questions 
were simultaneously varied, different results were obtained. Felker 
and Dapra (1975) found the effect of conceptual postquestions to be 
superior *d tha^ of concepcual prequcstions, while Rickards found 
-^•e oppor5ite t'> be true, and Mayer (1975) found no differential 

rect due to position. Presumably, these differences were in part 
a consequence of the different kinds of conceptual questions employed 
by each investigator. Accordingly, it is not yet possible to ascertain 
"which kinds of conceptual questions are likely to function best as 
prequestions and which as postquestions" (Rickards & Denner, 1978). 

Preqnestions and Postquestions Compared to No Qu est ions 

Sagaria and DiVesta (1978) tested the hypothesis that the effect 
of "pre- aid postquestions in combination (QBA) would be equivalent 
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to the combinod independent effects of prequestions only (QB) plus 
postquestions only (QA) They found that: (a) prequestions enhance 
selective attention but not selective retention; (b) QBA produces 
nore intentional learning than QB, but not more than QA; (c) in 
relation to incidental learning, QBA does not benefit more than QA 
or NoQ; (d) frequent and regular QA tends to increase intentional 
learning over time at the expense of incidental learning. These 
last results correspond to Ausubel's statement that **the more the 
material is broken by questions, the more incidental learning tends 
to be inhibited; whereas, irequent questions tend to improve retention 
of relevant (intentional) material" (Sagaria & DiVesta, 1978), 

Inferred Processes Associated with the Use of Adju ne t Questions 

The question raised by Frase about hypothetical backward or for- 
ward effects produced by questions inserted in prose has been 
frequently submitted to experimental research. With the design of 
different new techniques, several results have been obtained. For 
example, Mayer (1975), found an enhancement of transfer to new 
situations and assumed that inserted questions perform a specific 
forward function. McGraw and Gotelueschen (1972) found both general 
forward and backward processes associated with a temporary increase 
in attention to the whole passage. Later, Rickards and DiVesta (1974) 
found that: (a) factual postquestions alone promoted a specific 
backward process; (b) postquestions related to rote learning of ideas 
suggested to them a general backward process (review); (c) the 
analysis of the results under ireaningful learning poslquestion 
conditions tended to indicate a specific forward and backward process. 



In conclusion, it seems that adjunct questions control the 
acquisition of intentional and incidental information differently. 
As Sagaria and DiVesta stated, prequestions function as discrimina- 
tory cues. As compared to a no-questions condition, questions seera 
to direct attention toward information that would not otherwise be 
taken into account. When material is poorly organized, inserted 
questions exert a guiding influence; at the same time, positive 
feedback provides the information that the strategy adopted 
adequate and should be maintained. Sagaria and DiVesta (1978) 
hypothesized that inserted questions evoke expectancies that deter- 
mine the selection of specific learning strategies. On the other 
hand, postquest ions enhance the retention of material attended to in 
the encoding process. Although the joint action of both pre- and 
pos tquestions does not result in better learning of incidental infor- 
mation as compared to postquestions or no questions at all, it does 
enhance the selective retention of intentional material. 

Individual Differences in Relation to Inserted Questions 

An important finding for distance education is that experimental 
research has show that subjects with low verbal capacity benefit more 
from higher-order postquestions than subjects with high verbal 
capacity who achieved about equally under question and no-question 
conditions (Shavelson et al,, 1974). Rickards and Hatcher (1978), 
applying meaningful learning questions, rote-3earning questions, and 
no questions to fifth-grade children, found that poor comprehenders 
shov;ed significantly higher recall v;ith the first condition. They 
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concluded that the higher-lv»vel postquestions may not bensfit good 
comprehenders as much as poor comprehenders because good comprehenders 
probably generate a context spontaneously while reading. 

It is worthwhile citing the observation by Gustafson anci Toole 
(1970) that in normal homework assignments inserted questions may not 
have a general facilitative effect. Hiller (1974) tested the 
hypothesis that, in home study, students may have time to read and 
reread the material. His study confirmed this hypothesis and also 
found that students tend to study only the Inserted questions recalled 
to the material at hand but not the text as a whole. 

Pos tinstructional Strategies 
Learning from text is influenced by events that occur after the 
learner has studied the text material. Priirary among these events is 
feedback. 

Feedback 

Feedback consists in "any of the numerous procedures that are 
used to tell a learner if an instructional response is right or 
wrong" (Kulhavy, 1977). It ranges fr^m the simplest "Yes-No" form to 
the presentation of substantial information which may extent the 
content of the instructional material or even add new information 
to it. This latter ^o»t.. goes beyond simply informing about 
correc*'ness. 

Numerous studies have been conducted in order to show the efi.ects 
of feedback on learning. The results are sometimes positive and 
sometimes negative. Kulhavy, trying to explain these contradictory 
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results, revised the material and procedures used in these experiments. 
He identified two negative condif^ons under which feedback fails to 
perform its facllitatlve function* One is that if text formats permit 
learners to see the feedback before responding, copying is likely to 
result and students learn lictle because they write without searching 
through or even reading the text. In other words, they bypass most 
of the instruction (Anderson & Faust, 1967). This finding can be 
related to those of Gustafson and Toole (1970) and Hiller (197A), 
cited above in relation to adjunct questions in a text. Second, when 
the material studied is very difficult for the learners, they spend 
most of their time guessing at answers and then trying to associate 
the feedback with the question. Feedback performs its correcting 
function effectively only if mistakes result from faulty interprcca- 
tion, not from lack of understanding. If learners fijil to comprehend 
the material, feedback has no more than cursory effect on performance. 

Once decrcmental factors like answer availability and low com- 
prehension are accounted for, feedback performs tv;o functions. It 
confirms right answers, and it identifies and allows the learner to 
correct wrong ones. Feedback following wrong responses has the 
greatest positive effect. When learners have high response confidence 
and the response chosen is wrong, they are likely to spend a consider- 
able amount of time in trying to locate the source of their mistake. 
U^en response confidence is low, feedback has a minimal effect, regard- 
less of whether the answer is correct or not, since students with 
poor comprehension are likely to 'jse associative strategics rather than 
the integration of new information into an existing structure. 
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Kulhavy insisted that there is a powerful interaction between 
a learner's expectation and feedback. This interaction affects 
what the learner will remember from a text. Based on this finding, 
Kulhavy postulated that feedback and its form of presentation can 
be manipulated more effectively if they are in some way made con- 
tingent on learner expectation. For instructional design, his 
principal recommendations were the following. "First, make sure that 
learners have appropriate entry skills for the lesson; second, struc- 
ture the material in such a fashion that the response precedes the 
feedback, in spite of the student. Finally, provide feedback as often 
as possible during the course of the lesson'' (Kulhavy, 1977). 

^ Interaction of Strategies 

A number of studies have examined the effects of the strategies 
described above in interaction with one another. 

Learning Objectives and the Organization of Prose 
Rothkopf and his associates studied the interaction of objec- 
tives with both the density of the information in a text (Rothkopf & 
Kaplan, 1972) and with the degree of dispersion of the information 
in a text (Gagne & Rothkopf, 1975). 

Duell (197A) reached the conclusion that the importance attributed 
to a text element determines whether or not objectives will facilitate 
learning. If learners consider an element to be important, they will 
learn it, whether or not it is related to an objective; with an element 
judged less important, learnf.rs will It influenced by the presence or 
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nbsonce of an objective related to it. Melton (1978) came to a 
sir.ilar conclusion conducting a study in a nrtura] setting. 

Duchastel (1979) investigated how the structure of the text 
interacts with objectives on learning. There were no joint effects 
of structure and objectiv?s. This implies that the provision of the 
two forms of orientation add little to the provision of either one of 
them alone. Like previous researchers (Duell, 197A; Melton, 1978), 
Duchastell concluded that objectives are more likely to affect 
learning positively if they relate to material which is judged to be 
unimportant by the learners. 

Level of Adjunct Questions, Type of Feedback, 
and Learning Concepts from Prose 
The effects of level of questioning (factual questions or 
application questions) and the type of feedback on learning of con- 
cepts from prose were examined in a study of undergraduate students 
(Singer & Rhodes, 1976), Adjunct application questions produced 
better performance on subsequent new application items. This result 
supported the conclusion that practicing application while studying 
facilitates later use of concepts. Feedback influenced performdnce 
on the questions asked during the instruction, but not on new factual 
or application questions. These results underline the value of what 
students do during learning and the importance of their use of 
feedback. 
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Chapter III 
Method 
Subjects 

The subjects in this study were 42 volunteer college students 
enrolled in the Education Department of the Universidad Nacional 
Abierta (U.N. A.) in Caracas. They were candidates for admission 
to the program in Learning Disabilities which was scheduled to 
begin for them six months later. They therefore were not acquainted 
with the material they would be exposed to in this study (as was 
confirmed by their answers to the questionnaire which will be 
described below) . 

Materials 

Three kinds of materials were used in this study: 

- learning materials 
" a post test 

- a questionnaire 

Learning Materials 
The material selected for this study was one '^unit" from an 
introductory course on Learning Disabilities (Course: "Dif icultades 
de Aprendizaje I**). The content of this material was mainly a 
definition of learning disabilities and the functions performed by 
professionals in this area. Three versions of this material were 
prepared: 1. Interspersed; 2. Separated (segregated); 3. Eliminated 
(removed) . 
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1. The Interspersed Version was a copy of a "unit" taken from a 
text which is presently being used as instructional material at U.N. A. 
As mentioned in the first chapter, this material was div^iaed into 
three sections: introduction, instruction, and postinscruccion. 

The introductory section, which functioned as an advance 
organizer, included: 

(a) a table of contents, 

(b) an explanation of the task the student was exepcted to 
perform, 

(c) a general introduction to the field of learning disabilities. 
The instructional section included: 

(a) an introduction to the unit (advance organi^er), 

(b) the text itself, including definitions and examples, 

(c) exercises (interspersed postquestions plus feedback). 
The post instructional section consisted of: 

(a) a list of the general and specific objectives of the unit, 

(b) a self-evaluation, 

(c) feedback (acceptable answers and additional explanations). 
The actual U.N. A. text also contains a list of reference books 

and articles, but this was removed in order to find out — by means 
of the questionnaire — if the students were interested in looking for 
additional information after studying the unit. 

The length of the unit was 52 pages (approximately 13,^20 words) 
divided as follows: 
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Inwroductory section 



8 pages 



Instructional section 



35 paf,«s 



Title 



1 page 



Introduction 



1 page 



Instructional text 



18 pages 



Exercises 



15 pager, 



Postinstructional section 



9 pages 



The content of the unit was designed and written by a U.N. A. 
professor on the basis of current information about the field. 
Underlying the development of the material were several objectives — 
one general and six specific — which appeared at the beginning of 
the postinstructional section. 

2. The Separated (or segrated) Version was presented in two 
booklets. The first (Selected Readings) conveyed the instructional 
information. This was Uk pages long (approximately 21,968 words). 
This material corresponded in content to the "instructional text** of 
the first version. This text was not wrJtten by U.N. A. teachers, but 
was taken from several standard works in the field of learning dis- 
abilities. The readings were selected in order to cover the same 
subject ma,.ter as the respective section of the first version. They 
were presented in logical sequence in the booklet. 

The second booklet, the Study Guide, contained all of ihe 
instructional "aids.** These aids were exactly the same as those 
contained in the first version. In other words, the following elements 
were segregated from the first version and placed in this booklet: 




- introducto section, 

- exercises (questions plus feedback), 

- postinstructional section (objectives, self-evalua».ion, and 
answers) . 

3. The Eliminated (or removed) Version had the same content as 
the first booklet (Selected Readings) of the second version. All cI 
the instructional "aids" were totally eliminated, however. That is 
to say, the information was given in pages (approximately 13,968 
words) . 

Additional Comments with Regard to the Learning Material 

Each of the versions contained several distinctive features: 
The Interspersed Version, for example, was organized as described 
below: 

1. Introductory section; No objectives were presented at the 
beginning of the unit since it was argued that incidental learning 
should be encouraged rather than hindf-red. It was also argued that 
this important aim would be furthered by inserting some essay ques- 
tions into the text. These were intended to stimulate active coq- 
parisions, inferences, and the derivation of social or educational 
implications • It was also thought that a long list of operational 
objectives would overwhelm the student and reduce motivation. In 
addition it was believed that if too many objectives were specified, 
a number would not be read with comprehension by the students and 
they would therefore not fulfill their intended purpose. 

However, as mentioned before, one general and six specific 
objectives were written in order to develop the instructional 



design, the self-evaluation, and the posttest. This explicit list was 
included at the end of the instructional material, as an introduc- 
tion or guide for the self-evaluation. The intention was ro provide 
the students with a basis for review and to offer them an organiza- 
tional framework for that purpose. In this sense, the specific 
objectives had a teaching purpose, i.e., placed at thz end of the 
instructional material, they should influence rehearsal and produce 
long-term effects. Therefore, such objectives were assumed to 
function more as inserted questions than as objectives* 

With regard to the general information and the introduction to 
the unit, used as advance organizers, they were written in a very 
general manner, in an c ' empt to establish the relations!iip of the 
unit to the area of learning disabilities as a whole. 

In the presentation of the study — which appeared at the 
beginning of the unit — emphasis was placed on time scheduling and 
on the fact that students should participate as actively as possible 
in the learning process. They were asked to respond to all of the 
interspersed questions, to follow the explanations and examples, to 
take notes, to make summaries, or whatever symbolic representations 
would help them understand and review the material. 

2. Instructional section: This section was organized around 
maxii topics (general titles) which were in turn subdivided into 
component sections (subtitles). Also, as mentioned before, there 
was a table of contents to help students grasp the organiJ^ation of 
the unit. 



The style of this version was expository, and an effort was nade 
to establish a personal relationship with the student. Special care 
was taken to define each new concept and offer at least one example. 
The sequence of the presentation variel (definition plus example, or 
examples plus definition, or a definition followed by an exercise 
which was intended to serve as an example, etc.) in order to avoid 
monotony in style. The main id as were generally placed at the 
beginning or end of each paragraph; the remaining sentences usually 
were lower-level clarifications. There was a deliberate attempt to 
select examples based on the frequency with which problems are encoun- 
tered in the field of education in Venezuela. 

The exercises contained in the instructional section had several 
purposes, depending on where they were inserted. The main aims were 
to review or redefine concepts, to infer interrelationships, and to 
suggest implications. Since the course was an introductory one, the 
emphasis was on concepts and their clear demarcation. The nature of 
the questions varied in order to introduce no,elty and encourage dif- 
ferent levels of processing. Some questions were multiple-choice, 
others were completion, others were short essay questions^ and so on. 
Sometimes, mapping or summarizing was requested. 

Feedback was offered immediately after questioning, frequently 
accompanied by explanations which attempted to complement the infor- 
mation in the instructional body of text. When free answers were 
requested, the feedback gave a general outline of the points that it 
would have been advisable to include and developed some arguments when 
reasoning was requested. 
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3. The postinstructional section presented a list of objectives 
and a self-evaluation. The self-evaluation aimed at an overview of 
the principal aspects of the unit. It emphasized the main concepts 
and relationships. It was constructed following a table of specifi- 
v-ations, based on three levels of Bloom's taxonomy: information, 
comprehension-application, and synthesis. This approach, as mentioned 
in the first chapter, tends to correspond to DiVesta's view of the 
learning process (DiVesta, 1972). The test consisted of 33 simple- 
choice items. Feedback following the self-evaluation was also given in 
an expository form when special explanations were needed. In that 
sense, feedback was thought of as a teaching device and added new 
information to the text. 

The distinctive features of the Separated Version were as 
described below. 

1. The Selected Readings of ihe first booklet were a series of 
several fragments (quotations from books and articles) which were part 
of the current literature in the field of learning disabilities. The 
language used in this material was very different from the language 
used in U.N. A. 's instructional units v;here a deliberate attempt was 
made to use simple words and examples for each technical term. In 
spite of the logical sequence in which the text excerpts were placed 
in the booklet, what was missing was an internal organization based 
on an instructional design clarifying the content at an iiitroductory 
level. Nevertheless, a positive aspect of this material was that the 
information was given directly frorr, its sources, that is, from 
specialists in the field, and not through the language of a teacher. 
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2. The Study Guide contained all of the instructional 
directions telling the student when to read and when to do the 
exercises. It attempted to give the information gradually (as in the 
lnterspers2d Version) and to establish some kind of personal relation- 
ship with the student using the pronoun "you" and referring to the 
current situation in the country whenever possible. As stated before, 
the content of this booklet was exactly the same as "that of the 
"instructional aids" of the' Interspersed Version, but printed 
separately. 

The Eliminated Vers ion consisted of only one booklet; the same 
Selected Readings used for the Separated Version. In other words, 
this version contained the same information as the other two versions, 
but all instructional aid was removed. 

Post test 

One posttest was developed for the measurement of the dependent 
variable. It was constructed following a specifications table, based 
on the three levels defined in Chapter One. The same posttest vas 
administered to all students, no matter what version of the instruc- 
tional material they studied. Appendix A contains the Table of 
Specifications (with special reference to levels A, B, and C) , the 
Posttest, and the Answer Key. 

Questionnaire 

A questionnaire (Appendix B) was administered at the end of the 
study in order to clarify such variables as: 
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- educationa] background of the student 

- age of the student 

- professional experience of the student 

- time scheduling 

- study behavior with the instructional material 

- Incentives that were used to encourage study 

- use of objectives, advance organizers, instructions, examples, 
Inserted questions artd feedback, self-evaluation, and feedback 

- the perceived usefulness and clarity of the different parts of 
the material 

- interest in looking for additional information on the topic. 

Procedure 

The study actually started with 120 volunteers, to whom the pur- 
pose of the study was explained. After having agreed to participate, 
they were randomly assigned the study material. Thus, three groups 
were formed: 

Group 1 received the Interspersed Version 

Group 2 received the Separated Version 

Group 3 received the Eliminated Version 

When the posttest was administered, only 42 subjects returned 
(35 perc'^nt of the original sample). Each group was represented by 
14 subjects. The remaining subjects were called by phone, but it 
was impossible to get them to participate further. The data that 
were analyzed referred to the 42 subjects who took the posttest and 
filled out the questionnaire. 



To understand the study and the cooperation expected of them, 

the students received a general explanation in Spanish (see 

Appendix C.l) which said in part.: 

Since the Universidad Nacional Abierta is a new national 
project created as an attempt to improve the quality and 
quantity of higher education in Venezuela, the University 
needs to conduct experiments in the field of education 
in order to identify the best strategies for teaching and 
learning. You have been selected at random to participate 
in one of the studies we are conducting. The. purpose of 
this study is to determine the best presentation for our 
courses. We have prepared instructional material for you. 
You will study it at home, as best you can, and will come 

back with all your materials next . You will 

then receive further instructions. 

3. Two weeks after receipt of the material, the students were 
all assembled on the same day in three classrooms. They received 
some general information (see Appendix C.2) as an introduction to 
the session. Afterwards, they were administered the questionnaire 
(a different version for each group, since the questions all related 
to the instructional material and this differed from one group to the 
next). After completion of the questionnaire, each student was 
administered the posttest (the same for the three groups). At the end 
of the session, each student underwent a short interview with the 
researcher in order to express his feelings about the study. 

A pos ttest-only design with three groups was employed in this 
study. In order to avoid any facilitativc effect, especially in 
Group 3, no posttest was used. For each group, the design was the 
following: 
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Group 1 




Group 2 



R X 



2 



0. 



2 



Group 3 



R 0^ 



Data Analysis 

For hypothesis 1, an analysis of variance was employed to test 
the relationship between treatment and achievement. 



variance, the answers of the students to the questionnaire were 
analyzed. Several chi-square tests were used to find out whether 
there existed any relationship between the variables studied in the 
questionnaire and achievement. Three variables appeared to be 
important: study behavior, age, and educational background. 

For hypothesis 2, in which the purpose was to relate treattr.ent 
to study behavior, age, and educational background, three separate 
unweighted means analyses of variance were applied, Tliis technique 
was selected because of the unequal number of subjects appearing in 
each cell. 

For hypothesis 3, because the same subjects were measured at the 
three levels, a two-factor ANOVA with repeated measures on one-factor 
was employed. 



In order to better interpret the results of the analysis of 



Chapter IV 
RESULTS 
Hypothesis 1 

Stated in null terms the first hypothesis is as follows: 

Three versions of the same instructional written material 
will result in equal student achievement. 

The total score on the posttest obtained by each student in each 
experimental group is shown in Table 1, along with the group mean and 
standard deviation. 

TABLE 1 
SCORE OF EACH STUDENT 
ON THE POSTTEST 



Group 1 Group 2 Group 3 

(Interspersed Version) (Separated Version) (Kliminated Version) 

(N = 14) (N = 14) (N = 14) 



MEAN 
STANDARD 



33.0 


29.5 


18.0 


31.5 


26.0 


17.0 


31.5 


21.0 


16.0 


27.0 


19.0 


16.0 


24.5 


19.0 


15.5 


18.5 


16.0 


13.5 


18.0 


15.0 


12.5 


17.5 


14.5 


12.5 


15.0 


14.0 


12.0 


11.5 


14.0 


9.5 


9.0 


11.5 


9.5 


8.5 


11.5 


9.0 


8.5 


10.5 


7.5 


8.5 


8.5 


6.5 


18.75 


16. A 3 


12.50 


8.89 


5.74 


3.55 
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To test the first hypothesis, the data were analyzed using a 
one-way analysis of variance. A sunimary of the results is given in 
Table 2. The F-value for treatment (versions) is 3.2 (p - .0539). 
Although an F-value of 3.24 or larger is required for the mean dif- 
ferences to be significant at the .05 level., the obtained F-value 
is so close to the critical value that the first hypothesis was 
rejected, It was concluded ti^at student achievement was not the same 
under the three conditions of study. 



TABLE 2 

SUMMARY OF THE ANALYSIS OF VARIANCE FOR THE POSTTEST 
IN THE THREE GROUPS 





Source of Variance 


df 


ss 


ms 


V 


P 


Between groups 
Within groups 


2 
39 


27<>.A7 
17A5.80 


3.39.73 
^^.76 


3.12 


.06 



In order to determine where the difference in achievement actually 
o'lcurred, a Sheffe test was applied. The critical value for the dif- 
ference to be significant at the .05 level on the Scheffe test was 
5.71, Comparing M^ and M^, M^ and M^, and and M^, a significant 
difference (6.25) was found between groups 1 and 3 in achievcmenc. 
Although it is not statistically signi'^icant , the difference between 
M^ and (3.93) was higher than the difference between M^^ and 
(2.32). 
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To better interpret these results, the responses of each student 
to the questionnaire were examined. First, what the students did 
during the learning process was studied. Using chi-&quare tests, the 
three groups were compared on the basis of number of study sessions, 
time spent in studying, frequency of review, and frequency of con- 
sultation with other persons or use of bibliography. No relationships 
were found between these variables and the version under study. 
Nevertheless, when the groups were compared on the basis of "study 
behavior" (item 11*1, 11,2, 11,3, 11,4, 11,5, and 11.12 of the question- 
naire) a tendency to exhibit this type of behavior to a greater extent 
appeared in group 1 than in the other groups (p < .05). When 
groups 1 and 2 were compared regarding the frequency with which they 

carried out the exercises (items 11.6 and 11,7 of the questionnaire), 

2 

significant values were found (x = 8.90, df. 2, p < .02), indicating 
that group 1 was more active in this respect. 

Student opinions about what helped them most in their study 
(items 16.1,1,1 to 16.1,1,11 of the questionnaire) were also analyzed. 
A significant difference appeared between groups 1 and 2, showing that 
group 1 considered that all of the instructional aids (items 16.1.1.9 
and 16.1,1,10) has been of much help in their learning, except for 
the illustrations and the additional comments. Group 2 had more 
trequent answers under "Somewhat" (the instructional aids had ''help^Pd 
somewhat" in their learning). In the questions that were also shared 
by group 3 (items 16,1,1,3, 16.1,1,4, 16.1,1,8, and 16.1,1.11: 
organization of the text, explanations inserted in the text, style of 
the text, arid "all of the above"). Croup 3 showed a tendency similar 
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to that of group 1, that is, a tendency to indicate that these 
instructional characteristics had been of much help in their learning 
(the level of significance for item 16.1.1.8, for example, was 
p < .01). 

Questions 20 and 21 of the questionnaire were open-ended questions 
directed toward the incentives that were used by the study to keep 
studying the material and the student's feeling toward the material 
and the field of learning disabilities. The answers given by the 
three groups, in terms of frequency, are given in Table 3. 

The aspects that were criticized by the students also varied 
between groups, as shown in Table 

Table 5 summarizes the answers of the students to the first part 
of item 30 of the questionnaire: positive aspects of the material. 
This question was placed in the questionnaire in order to compare the 
answers to those given to questions 20 and 21 that were worded in 
terms of the feelings of the learner (pleasant-unpleasant). 
Question 30 was worded in more objective terms: what was negative in 
the material? 

Table 6 summarizes the answers of the students to the second part 
of item 30 of the questionnaire which referred to the negative aspects 
they found in the instructional material. This question has the aim 
of checking the answers to question 21, and it was expected that the 
responses would show a similar tendency as question 21. In fact, it 
added some new information in the same direction. 
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TABLE 3 

FREQUENCY WITH WHICH SEVERAL ASPECTS OF THE MATERIAL 
WERE APPRECIATED BY THE STUDENTS 

(Baced on Questionnaire Item 20) 



ASPECTS 



GROUP 




12 3 4 


5 6 7 8 9 10 11 12 13 K 15 16 


Group 


1 


2 2 1 


2 2 2 2 1 1 2 


Group 


0 


1 


2 2 13 2 


Group 


i 




2 2 2 2 2 


Aspect 


1: 


The material 
society 


gives me hope that I will be useful to 


Aspect 


2: 


The material 


uses different approaches 


Aspect 


3: 


The material 


is absorbing (very interesting) 


Aspect 


4: 


Examples are 


given at each level cf information 


Aspect 


5: 


The material 


is not tiring or boring 


Aspect 


6: 


The material 


shows the relp'"' to experience 


Aspect 


7: 


The material 


provides counstixng about my career 


Aspect 


8: 


This is a dynamic way of helping the student to improve 


Aspect 


9: 


The wording 


is pleasant 


Aspect 


10: 


The material 


is simple and clear 


Aspect 


11: 


The material 
planning 


shows the importance of diagnosing and 


Aspect 


12: 


The material 


shows cases and motives 


Aspect 


13: 


The material 


Introduces one to research 


Aspect 


14: 


The material 


expiairb how to manage children 


Aspect 


15: 


The material 
and learning 


gives information on the nervous system 


Aspect 


16: 


All the material 
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GROUP 

Group 1 
Group 2 
Group 3 



TABLE A 

FREQUENCY WITH 'THICK SEVERAL ASPECTS OF THE 
MATERIAL WERE CRITICIZED BY THE STUDENTS 
(Based on Questionnaire Icem 2i) 



3 1 
A 2 
A 



ASPECTS 



Aspect 1: Some aspects of the material disliked, but not described 
specifically 

Aspect 2: The material was a photocopy 

Aspect 3: The kind of printing 

Aspect A: Position of the objectives 

Aspect 5: Lack of the use of color in the material 

Aspect 6: Too many references and citations 

Aspect 7: Specialized vocabulary 



GROUP 

Group 1 
Group 2 
Group 3 



TABLE 5 

FREQUECNY OF COMMENTS ON POSITIVE ASPECTS OF THE MATERIAL 
(Based on Questionnaire Item 30) 



ASPECTS 



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 



6 6 4 I 1 I 2 3 3 
2 2 2 2 

5 5 2 1 3 



1 4 5 3 3 3 

2 2 2 2 2 
2 I 4 



3 
2 



Aspect 


1 


: Pleasant and simple wording 


Aspect 


13: 


Aspect 


2 


: Quality of the exercises 






Aspect 


3 


: It stimulates the desire to study 


Aspec t 


14: 


Aspect 


4 


: It gives information about special 


Aspect 


15: 






education 


Aspect 


16: 


Aspect 


5 


: The comparison between different 


Aspect 


17: 






approaches (the way they are compared) 


Aspect 


18: 


Aspect 


6 


: Complementary notes (final section of 
the instructional material) 


Aspect 


19: 


Aspect 


7 


Role of the specialist (or teacher) in 
learning disabilities 


Aspect 


20: 


Ai;pec t 


8 


Organization of the material 


Aspect 


21: 


Aspect 


9: 


Self-evaluation 






Avmct 


10 


The scienctific approach 


Aspect 


22: 


Aspect 


11 


Preciseness 


Aspect 


23: 


Aspec t 


12 


Abundant exercises 


Aspect 


24: 



Interesting information, clearly 
explained 

Orientation about my career 

Examples 

Short in length 

Selected Readings 

Final synthesis 

Figures (they are helpful for the 
explanations ) 

Study Guide (it allows review and 

sel f -correct ion) 

Brings awareness about these 

problems and education 

Easy to study 

Clari f ies doubts 

Provides basic concepts 
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TABLE 6 

FREQUECNY OF COffl-EKTS ON NEGATIVE ASPECTS OF THE MATERIAL 
(Based on /uesfcionnaire Item 30) 



ASPECTS 

GROUP 1 2 3 4 5 6 7 8 9 10 ?1 12 13 14 15 16 17 18 

Group 1 1 3 4 2 5 1 1 1 

Group 2 11 "111 

Group 3 1 4 14111111 



Aspect 


1: 


Place on the objectives (at the end) 


Aspect 


2: 


It is a photoccs-; 


Aspect 


3: 


Lack of illustrations 


Aspect 


4: 


Letter print 


Aspect 


5 : 


Lack of references 


Asepct 


6: 


cabuiary too technical 


Aspect 


7: 


Lack of the use of color in the material 


Aspect 


8: 


Repetitive 


Aspect 


9: 


The original texts are in English 


Aspect 


10: 


Too many citations 


Aspect 


11: 


Too short 


Aspect 


12: 


Too general 


Aspect 


13: 


Con fus ing 


Aspect 


14: 


Lack of a summaiy at the end 


Aspect 


15: 


Lack of self-evaluation 


Aspect 


16: 


Lack of a glossary 


Aspect 


17: 


Two booklets (one would be bet*'e 


Aspect 


18: 


Selected Read*-:go, difficlt 



Hypothesis 2 

v,i:ov5 in null terms, hypothesis 2 is as follows; 

: ^-o versions of the same instructional v;ritten material 
:,-.».lts in equal scudent achievement >-egardlcss of: (a) 
n;..»:v behavior, (b) age, and (c)s edu^ ^t ional background. 

Hypothesis 2. a 

\-rlicd specifically to the study behavior of the student, 
: -.oiiis 2 may be stated as follows: 

Trree versions of the same instructional written material 
results in equal student achievement regardless of study 
rehavior . 

Study behavior implies doing items 11.1, 11.2, 11.3, 11. A, 11.5, 
.I'j 11.12 of the questionnaire, "frequently," sometimes," or "never." 

ccher words, it is based on the frequency with which students 
•':.:'t chat they reread principal parts, take notes, make diagrams, 

; follow the indicated sequence. Table 7 shows the means and 
:j::dard deviations of study scores on the post test, based on 
-.jvior and treatment group. 

An analysis of variance was applied to these data; since the 
^lls had different numbers of subjects, an unweighted means analysis 
• ^ or.ployed. Table 8 shows the summary of the results of this 
' lysis. 

The F~ratios for study behavior and the study behavior by 
'* 'ilr.cznt group interaction are not statistically significant. 
• '-refore, the null hypothesis is retained. There is no significant 
••••oronce in student achievement due to study behavior and there is 
*-'/lc]ence of an interaction effect between study behavior and 
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TABLE 7 

rOSTTEST MEANS AND STANDARD DEVIATIONS OF STUDENT GROUPS CLASSIFIED 
ACCORDING TO TWO VARIABLES: TREATMENT GROUP AND STUDY BEHAVIOR 



Treatment Group 



Study 1 2 3 All 

behavior Interspersed Separated Elindnated Groups 

Responses Version Version Version 



Mostly 


Mean 


21.33 


16.50 


7 


.83 


17 


.67 


"Frequently 


" S.D. 


10.46 


2.29 


1 


.53 


9 


.65 


Mostly 


Mean 


13. 12 


20.20 


14 


.00 


15 


.62 


"Sometimes" 


S.D. 


3.94 


7.49 


?. 


.56 


5 


.41 


Mostly 


Mean 


18.00 


13.24 


13 


17 


13 


.70 


"Never" 


S.D. 


0.00 


4.34 


4 


54 




. 16 








TABLE 8 












ANALYSIS 


OF VARIANCr. 


OF POSTTEST 


SCORES 


BASED 


ON 





STUDY BEHAVIOR AND TREATMENT GORUP 
(Unweighted Means Analysis) 



Source of Variation 


df 


ss 


ms 




F 


P 


Study Behavior (A) 


2 


4.477 


2.238 


0 


.055 




Treatment Group )B) 


2 


187.392 


43.696 


2 


304 


.114 


X B 


4 


249.986 


62.496 


1 


.537 


.213 


Within Cells (Error) 


33 


1342.196 


40.673 
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Hypothesis 2.b 

Applied specifically to the age of the student, hypothesis 2 
may be stated as follows: 

Three versions of the same instructional written material 
results in equal student achievement regardless of age. 

Students were assigned to two age groups; the means and standard 
deviations of student scores, based on age and treatment group, appear 
in Table 9. The analysis of variance of these d.itj, based oi^ an 
unweighted means analysis, is summarized in Table 10. 

The F-ratios for age and the age by treatment group interaction 
are not statistically significant. Therefore, the null hypothesis 
is retained* There is no significant difference in study achievement 
based on age and there is no evidence of an ;jgc by troatmenl 
interaction. 

Hypothesis 2 . c 

Applied specifically to the eduL.Jtional background of die 

students, hypothesis 2 may be stated as follows: 

Three versions of the same instructional written material 
results in equal student achievement regardless of educa- 
t ional background . 

The mean and standard deviations of the student scores, based 
on educational level attained and treatment group, are presented in 
Table 11. 

An unweighted means analysis was applied to these data. Table 12 
presents the summary of the results of this analysis. 

The F-ratios for educational level and the educational level by 
treatment group interaction are not statistically significant. 
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TABLE 9 

POSTTEST MEANS AND STANDARD DEVIATIONS OF STUDENT GROUPS CLASSIFIED 
ACCORDING TO TWO VARIABLES: TREATMENT GROUP AND AGE 



Age 
Group 
(in years) 



20 to 30 



30 and 
above 



Mean 
S.D. 

Mean 
S.D. 



Treatment Group 



17.69 
13.17 

20.17 
9.05 



17.58 
5.94 

15.56 
6.22 



^13. 14 
3.18 

11.30 
4.59 



1 2 3 Al. 

Interspersed Separated Eliminated Groups 
Version Version Version 



15.89 
6.89 

15.89 
7.38 



TABLE 10 

ANALYSIS OF VARIANCE OF POSTTEST SCORES BASED ON 
AGE AND TREATMENT GROUP 





(Unwei 


ghted Means 


Analysis ) 






Source of Variation 


df 


ss 


ms 


P 


P 


Age (A) 


1 


2.214 


2.217 


.047 




Treatment Group (B) 


2 


309.378 


154.689 


3.277 


.048 


A X B 


2 


43.781 


21.891 


.464 




Within Cells (Error) 


36 


1699.529 


47.209 
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TABLE 11 

POSTTEST MEANS AND STANDARD DEVIATIONS OF STUDENT GROUPS CLASSIFIED 
ACCORDING TO TWO VARIABLES: TREATMENT GROUP AND EDUCATIONAL LEVEL 



Educational 
Level 



Treatment Group 



1 2 
Interspersed Separated 



Eliminated 



All 
Groups 



High 


Mean 


17.36 


12.60 


12.60 


14 


56 


School 


S.D. 


8.52 


2.63 


3.38 


6 


13 


Technical 


Mean 


11.75 


18.00 


11.67 


14 


06 


School 


S.D. 


A. 60 


1.73 


4.01 


4. 


37 


Superior 


Mean 


23.50 


18.83 


12.83 


i?. 


09 


School 


S.D. 


10.34 


7.90 


4. 38 


8. 


48 



TABLE 12 

ANALYSIS OF VARIANCE OF POSTTEST SCORES BASED ON 
EDUCATIONAL LEVEL AND TREATMENT GROUP 
(Unweighted Means Analysis) 



Source of Variation 


df 


ss 


ms 




F 


P 


Educational Level (A) 


2 


155. 


767 


77.884 


1 


.832 


. 174 


Treatment Group (B) 


2 


179. 


528 


89.764 


2 


111 


.135 


A X B 


4 


216. 


391 


54.098 


1 


272 


.300 


Within Cells (Error) 


33 


1403. 


215 


42.522 
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Therefore, the null hypothesis is retained. There is no significant 
difference in student achievement based on educational level and 
there is no evidence of an educational level by treatment interaction. 

Hypothesis 3 

Stated in null terms, hypothesis 3 may be stated as follows: 

Three versions of the same instructional written material 
will result in equal levels of achievement of test items 
requiring different levels of processing. 

As described in Chapter 1, the posttest contained items requiring 
three levels of processing. The means and standard deviations of 
levels A, B, and C are shown in Table 13. 

Since each subject in each group was observed under three con- 
ditions corresponding to the three levels of questioning, a two-factor 
analysis of variance design with repeated measures on one-factor was 
used to analyze the data. The results are presented in Table lA . 

Since the data regarding trcatrient group are the same as the data 
analyzed in Table 2, it is not surprising that the F-ratio for the 
treatment main effect is exactly as reported in thdt table. The main 
effect for processing level is highly significant. An examination of 
the means indicates that students earned the lowest scores on items 
requiring deep processing. The interaction of treatment group and 
processing level is also statistically significant. This relationship 
is presented graphically in Figure 1. 
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TABLE 13 

POSTTEST MEANS AND STANDARD DEVIATIONS CLASSIFIED ON THE BASIS OF 
TREATMENT GROUP AND LEVEL OF PROCESSING REQUIRED 



Treatment Group 



Level 
of 



1 2 3 All 

Interspersed Separated Lliminated Groups 



Process ing 




Version 


Vers ion 


Ver-^ion 






A. Minimum 


Mean 


7.75 


5.39 


5.82 


6. 


32 




S.D. 


3.82 


2.00 


2.03 


2. 


88 


B. Moderate 


Mean 


8. 14 


8.54 


5.36 


7. 


34 




S.D. 


3.39 


2.72 


2.21 


3. 


10 


C. Deep 


Mean 


2.86 


2.50 


1.32 


2. 


23 




S.D. 


2.90 


1.93 


1.12 


2. 


17 



TABLE 14 

ANALYSIS OF VARIANCE OF POSTTEST SCORES BASED ON 
PROCESSING LEVEL AND TREATMENT GROUP 
(Repeated Measures Analysis) 



Source of Variation 



Between Subjects 

Treatment Group (A) 

Error Between 
Within Subjects 

Processing Level (B) 

A X B 

Error V/ithin 
Total 



df 



ms 



2 93.155 46.477 3.122 .054 

39 5S1.935 14.921 

2 616.333 308.167 121.709 .001 

4 53.155 13.289 5.248 .001 

78 197.512 2.532 

125 1542.089 
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Chapter V 
DISCUSSION 

This study tested the general >iypothcsis that there would be a 
difference in the achievement attained by college students working 
with different forms of the same instructional material. It was 
assumed that a group receiving an organized text containing all the 
instructional aids that an instructional design demands (Interspersed 
Version) would have a higher achievement mean than a group receiving 
the same information without any instructional aids (Eliminated 
Version) and a group receiving the same information together with 
the instructional aids printed apart (Separated Version). 

Analyses of the results obtained by the students on the posttcst 
indicated that Group 1 (Interspersed Version) performed at a signifi- 
cantly higher level than Group 3 (Hliminaied Version). Croup 2 
(Separated Version) showed achievement which stood in between those 
attained by Groups 1 and 3, but the differences found wore not 
statistically significant. 

The significant difference between Groups 1 and 3 supports 
numerous studies that have demonstrated the importance of instruc- 
tional aids. Nevertheless, if attention is turned toward the compari- 
son between and M^, and if it is taken into consideration that 
Group 2 had exactly the same instructional aids as Group 1, an explana- 
tion about why M2 - is not significant must be found. The question 
to be posed is therefore: what did the Interspersed Version have that 
the Separated Version did not, or what made - 11^ statistically 
significant? On comparing the Interspersed Version with the Separat^^d 
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Version, it appears that they differed in several ways. The Inter- 
spersed Version was arranged according to complexity for instructional 
purposes, all new or technical terms were defined as soon as they 
appeared within the text, examples and exercises relating the parts 
were immediately linked to the explanations, and the information was 
given in a short and condensed form. In contrast, the Separated 
Version was presented in two booklets (Selected Readings and Study 
Guide); hence, the learner had to expend a certain amount of effort 
to integrate both parts. Many technical terms were not defined, 
since the texts in the selected readings were not written for 
beginners but for professionals and the examples were not given at 
each step of the explanation as in the Interspersed Version. The 
effects of these last two aspects were intended to be reduced by me^ns 
of the Study Guide, since the main technical terms were defined there 
and examples were inserted as well, in the same terms as in the 
Interspersed Version, It seems, therefore, that the main differences 
consisted in the organized, graduated, linked sequence of information 
that existed in the Interspersed Version. This tends to confirm what 
has already been domonstrated in other settings, i.e., text organiza- 
tion and recall are positively related (Johnson, 1970; Meyer, 1975). 

In addition to the characteristics of the different instructional 
materials, factors related to the study behavior of the learners may 
be examined. One of these factors was what the students did to 
facilitate their learning. The answers to questionnaire items 11.1 to 
11.5 and 11.12 showed that although the same tasks were asked of 
Group? 1 and 2, students in Group 2 were not as active as students in 
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Group 1. Students in Group 2 did not report that they had reread the 
main parts, taken notes, made graphs, summarized, underlined, and 
followed the sequence of the text as often as students in Group 1. 
An examination of Tables 3, A, 5, and 6 are helpful in interpreting 
these differences. Although Tables A and 6 do not shed light upon 
what could have affected Group 2 negatively. Tables 3 and 5 show that 
the frequency with which students reported that the material was 
pleasant, interesting, or motivating (questionnaire item 20, aspect? 
3, 5, 9, 10, and 12 and questionnaire item 30, aspects 1, 3, and 13) was 
higher in Group 1 than in Group 2. The quality and the quantity of 
exercises were also more appreciated (questionnaire item 30, aspects 
2, 9, and 12) in Group 1 than in Group 2. Relating these observations 
to the fact that Group 3 which had no instructional aid at all also 
appeared more active than Group 2, it is plausible that the effort 
needed to integrate the two booklets produced n ijountermolivational 
effect. This effort might have discouraged the Group 2 learner from 
following the orientations given in the Study Guide. 

With regard to question 29 which asked the students if they felt 
any desire to quit, no subject answered affirmatively and, at the end 
of the questionnaire (item 3i), 70 percent of them said th^.t they were 
happy to have been enrolled in the study, and that they would be 
interested in repeating a similar experience. Item 28 revealed that 
no subject felt the study to be too long. The data may indicate that 
distance education students need to feel that they are part of the 
university, that their opinions are taken into account, and that they 
may share experiences with others. In effect, distance education 
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students have* little contact with professors; they study alone and 
only get together with peers when they are able to form a group on 



students on their satisif action for having been enrolled by the 
university in the study* In addition, the aspects of the material 
that were appreciated by the students (Tables 3 and 5) indicate that 
they were willing to study contents related to their lives and to 
practical matters (quastionnaire item 20, aspects 7, lA, 15, and 
questionnaire item 30, aspects A, 7, 10, lA, 21, and 2A) . 

In conclusion, organization, condensation, together with a 
certain kind of content that met the interests of adult learners 
were the factors which were most important for these students. The 
version studied also affected the behavior of the students during the 
learning situation; the activities of the students, what they did in 
the learning situation, was encouraped or inhibited according to the 
version to which they were assigned. Moreover, an examination of the 
responses to the questionnaire points to the fact that the versions 
also influenced motivation. 

The second hypothesis pointed to the question of whether some 
types of instructional materials were mora effective in relation to 
study behavior, age, and educational background. The results did not 
demonstrate any significant interaction between achievement and study 
behavior. Nevertheless, certain tendencies could be observed. In 
Group 1, the best achievement was attained by those who reported 
"frequent" study behavior and the worst by those who reported "some" 
application of study behavior. In Group 2, the best results were 
obtained by subjects who "sometimes" carried out tUe study behavior 
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and the worst by thc?e who "never'' applied it. V^;ith respect to Group 3, 
the highest scores were also attained with a non-frequent study 
behavior ^''sometimes'') and the lowest with no study behavior at all 
("never"). 

It c-^n be inferred that in the Interspersed Version, where the 
interspersed questions with their feedback were an integral part of 
the instructional design, studying the material without carying out 
the recommended activities would probably have led to the missing 
of some important information since questions were used as an instruc- 
tio'-a' dcivice and not as a checking device. Cn the contrary, in the 
Segtegated Vr.rsion, the carrying out or the recommcndeu activities 
demanded frequent interruptions in the reading of the Selected 
Readings. Stopping to read often may have had cwo effects: 
(a) students Secame discouraged and distracted and (b) learning 
became more difficult. In spite of this, students of Group 2 attained 
their best mean scores when they carried out the study behavior. 
Curiously, in tha E]iminated Version, students who "frequently" applied 
study techniques had a noticeably lower achievement than those who 
"sometimes" practiced the study behavior. This raises the question 
of what would have been the case for Gr^ ips 1 and 2 if they had nad 
no orientation or enhancement as to what -ictivity to perform while 
stud^ ig. The tendency in all groups was to avoid study techniques, 
sinv:e the total number of subjeccs who reported that tliey had "nevei 
or "sometimes" used them, in all groups, was greater than tlie total 
number of subjects who reported that they had "frequently" used them. 
However, as mentioned above, achievement tended to be belter among the 
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students to whom the «5tudy techniques were suggested and explained in 
the instructional material. Further studies are needed to diagnose 
what study techniques are actually employed by college students in 
Venezuela and how effeCive they ar^ 

With regard t. ..ge and educational level as related to achieve- 
ment, no statistically significant relations were evidenced. Never- 
theless, an examination of Table 11 indicates that there was a 
tendency among students of higher educaliondl levels in Groups 1 and 
2 to obtain better scores than stodents of lower educational levels. 
In Group 3, however, no tendency could be observed in achievement as 
relr'ed to educational level. Forther studies are needed to deter- 
mine whether studeiits of higher educational levels can better profit 
from instructional guidelines than lower educational level students. 

This third hypothesis had the purpose of clarifying whether the 
instructional design could enhance a higher level of processing in 
the learner. Therefore, the posttest was constructed on the basis 
of three levels of processing as defined in Chapter I. Although all 
s tuden^s obtained the lowes t scores on itemb requiring deep processing. 
Groups 1 and 2 attained a higher mean score than Group 3. In addi- 
tion, an interaction or treatment group and processing leval was 
found. This confirms Greeno's findings that text materials which 
emphasize relationships among the parts of a structure, . o ^he 
Interspersed and Separated Versions did tend to facilitate the 
learning of problem-solving strategies. Moreover, when levels of 
processing <re related to what students reported they did while 
studying, mathemagenic behavior is apparent. The amount and type 
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of activity during the learning process may determine the level of 
processing the leari^er will reach with specific information. 

Further studies with a greater sample of subjects are needed to 
confirm more precisely the results obtained in this study. Investiga- 
tions must be undertaken in order to determine which of the variables 
that form par** of an instructional design are actually essential ajid 
which interrelations among these variables have to be avoided in order 
to insure more effective learning. This study, nevertheless, demon- 
strated that different presentaiiops of instructional written material 
affect learning in different manners and that deep processing, an 
important goal for higher education, can be enhanced by instructional 
strategies. These facts should be taken into consideration hy 
teachers and administrators of distance education to support their 
derisions on what kinds of instructional materials should be chosen 
in order to meet the instructional objectives. 
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APPl'NDIX A 
POSTTEST 

This appendix preoents a translated version of the 
Posttesl administered to all of the participants in this 
study. 
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POSTTEST 



DIRECTIONS: 



a) Read each of the following statements. If a statement is true, 
circle the "T.'' If a statement is false, circle the "F." 



b) If a statement is false, change the underlined word (or words) 
to make the statement true. Place the new word in the blank 
space after the "F/* 

Example : 



I T F 



T F 



(matter) 



1. Nutriments are any kind of substance 
from which living beings may obtain 
wa^-er and energy. 



2. A 



La t ion is a set of individuals 



of the same species who live in a 
fixed place. 



T F 



T F 



T F 



T F 



T F 



T F 



T F 



1. According to the Directorate of 
Special Education, nonspeci fic 
learning disabilities are psycho- 
neurological disorders. 

2. In Venezuela v;e differcniate specific 
learning disabilities from nonspecific 
learning disabilities, according to 
Johnson and Myklebust . 

-3. Since the cause of specific learning 

disabilities is unknown, it is frequently 
said that they are psychoneurological . 

4. The brain has t: e property of trans - 
duction , which is the properly of being 
able to transform one kind of energy or 
information into another. 

5. A synonym for generalized disability 
is pedagogical retardation. 

6. Definitions of specific learning 
disabilities that have rm etiological 
approach arc those which point out the 
behavio r present in the disturbance. 

7. Specific learning disabilities are due to 
factors external to the individur^l. 
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T F 8, VJhen school performance is low due t.o 

sociocultural disadvantages it is 
generally referred to as global learning 
disability. 

T F 9. When people function inLolloi'Liuiliy 

below the population average due to 
sociocuiturai disadvantages, they are 
referred to as mentally retarde d ♦ 

T F 10. The Directorate of Special Education is 

especially interested in assisting indi- 
viduals with learning disabilities in 
nonverbal subjects. 



DIRECTIONS: 

Read the following description and answer questions 11 and 12. 



Angela S., 10 years old, is finishing second grade. 
Reference Reason: 



She reads as a pupil at Ihe first 
grade level. 



Physical Examination: 



General condition, adequate, 
height . 



Low 



Evolutive Development: Nothing special. Originates from 

very low socioeconomic background. 
Parents are frequently absent. 
She takes care of her younger 
brothers. 



School Antecedents : 



Psychological Study ; 



Absenteeism. Low achievement in 
all school subjects, except in 
gymnastics and drawing. The school 
has often changed the teachers in 
her classroom. 

I.Q. 98; some agression. 



11. This general description docs not allow us to diagnose the case. 
Nevertheless > with the available antecedents, we may suppose 
that — once the study finished — we will probably find some 
causal relationships that vill be able to explain Angela^s 



95 



low level of achievement. These casual relationships will 
probably be found between the pupil's achievoment and 

A. her emotional problems 

B. her low intellect:ual level 

C. her sociocultural disadvantages 

D. her motor integrity 

12. According to these few data, we may draw a prior — tentative — 
hypothesis that Angela's case is a case of a probable 

A. specific learning disability 

B. nonspecific learning c'sability 

C. psychonf»urological learning disorder 

D. mental retardation 

DIRECTIONS: 

Circle the option which completes or answers questions ]3 to 19. 

13. Specific learning disabilities have been defined in many dif- 
ferent ways according to different approaches. The authors who 
adopt an educational approach define S.L.D. 

A. emphasizing the cause of the disturbed behavior of L.D. 
children 

B. bounding their definition to an as objective as possible 
description of the essential characteristics of L.D. children 

C. describing ,ne neurological processes which are disturbed in 
L.D. children 

D. eliminating psychoneurological etiologies and referring prima- 
rily to the emotional disturbances existing in L.D. children 

lA. The fact that the nervous system is organized in subsystems 
which, occasionally, function in a semi-autonomous manner, 
would partly explain the 

A. idiopathic nature of specific learning disabilirie« 

B. etiology of specific ler-rning disabilities 



C. difficulties that characterize the treatment of learning 
disabilities 

D, specificity of the specific learning disabilities 

15. From the point of view of an educatoi , specialised in L.D., the 
most useful case descriptions for remedial programming are those 
which indicate the 

A. location of the brain damage of the L.D. individual 

B. set of behavioral symptoms of the L.D. individual 

C. relationships between the brain damage and the learning 
disturbance 

D. location and extension of Che crain damage of the L.D. pupil 

16. In our country, if we compare the incidence of Che cases of low 
school achievement due to specific learning disabilities with 
the incidence of the cases of low school achievement due to 
global disabilities, we find that 

A. we do not yet have data which can provide some indicators 
about incidence (higher or lower) 

B. it is of undetermined quantity, but probably higher 

C. informal observations done in Special Classrooms allow us 
to suppose that the incidence is lower 

D. informal observation*^ done in Special Classrooms allow us 
to suppose that the incidence is higher 

17. The following list gives characteristics that some pupils may 
present : 

1) normal inlelligcnce 

2) superior intell igence 

3) mental retardation 

A) school achievement unexpectedly low when compared with 
their capacity 

5) deficient cultural background 

6) severe sensory loss 

7) severe emotional disturbance 
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Which of the above characteristics are those which cannot be 
prominent in a case of specific learning disability, according 
to Bateinan? 

A. 1-2-3-4 

B. 2-A-5-6 

C. 3-5-6-7 

D. A-5-1-3 

The following graph shows the distribution of the I.Q. that 
people of any population present when they are administered 
an intelligence test. 

f 99 . 78% 
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Intelligence 
Quotients 



Using the definition of specific learning disabilities given by 
the Directorate of Special Education^ we may say that children 
with specific learning disabilities can be found 

A. between the limits pointed out by letters A and B 

B. between the limits pointed out by letters C and D 

C. between the limits pointed out by letters E and F 

D. at point G 

Mrs. Figueroa, a teacher of third grade, believes that ll-ye> -old 
Juana 0., of normal intelligence, has specific learning disabili- 
ties becauSw 

- Juana reads with a retardation of two years when compared with 
her fellow classmaces 

- Juana presents severe emotional disturbance which seems to 
provoke the learning problem 
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- Juana has not suffered from any cultural deprivation: her 
parents are middle class professionals and she has received 
conventional instruction sip'^e she was five years old. 

Taking as a reference point the definition of $'pecific learning 
disabilities of the Director-^ite of Special Education, we find 
that this teacher's point of view is 

A, correct, because discrepancy is n prominent characteristic 
of this pupil 

B. possibly correct, because the pupil has been receiving 
school attention for a long time 

C, incorrect, because. the teacher assumes that the cause 
of the disability is emotional 

D. possibly incorrect, because the pupil presents a normal 
intellectual level 

20, The word "psychoneurological" applied to learning problems 
indicates that the 

A. psychological factors produce a disturbance in behavior 

B. behavior of the individual is based upon a neurological 
substratum 

C. problems — of a psychological nature — produce 
neurological disturbances 

D. behavioral problems derive from neurological disturbances 

21. Among the data of a 10-year-old boy pupil, examined by the 
Psychopedagogical Service because of achievement problems, the 
following are prominent: 



Age: 


10 years old 


Reference 
Reason : 


low achievement in arithmetic 


1.Q-: 


110 


Evolutive 
Development : 


ability to sit, to grasp, to walk, 
and language development: normal 


Conclusion: 


specific learning disability in 
arithmetic 
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The conclusion is incorrect'. We know that in order to give a 
conclusion of this kind ve need basic supportive data. In this 
case, the Service was lacking enough data to reach that 
conc^Msion. 

Read the following antecedents. Circle those which, in your 
opinion, were missing to support the conclusion: 

A, measurement of the actual achievement of the pupil in 
arithmetic 

B, indication of his mental ability 

C, history of his physical and behavioral development 

D, sociocultural environment where he grew up 

E, grade he is attending 

F, information about his sensory conditions 

The following graph outlines the process by which the organism 
processes the information it receives from the environment. 



(a) 




(b) 


(c) 






Processing 






Sensory 
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(storage, 




Performance 
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codification, 
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recuperation) 
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Feedback 



Some of the parts of this process may be observed by the profes- 
sional and some can only be inferred. Identify which parts — 
described by letters a, b, c, and d — may be directly observed 
by the professional: 

A. a-b-c-d 

B. a - b - d 

C. a - c 



D. c 




DIRECTIONS: 



Study the following description and answer questions 23 to 28; circle 
one option which best completes or answers the statement. 



In a school of Caracas, the fourth grade teacher refers 
Andres to the Psychopedagogical Service; she asks for a 
specialized study because she would like to know how tc 
help Andres in reading. From the teacher's point of 
view, Andres is a 10-year-old boy, "smart,*' with an 
adequate general achievement, but with a reading level 
which actually corresponds to second grade. He makes a 
lot of mistakes and evidences a lot of hesitation when 
reading; for example, he frequently confounds letters 
'*a*' and **e,'* %" and "d/' **p" and "q." 



The Psychopedagogical Service, after several studies, 
concludes that: 

- Andres' I.Q. is 115 

- he has a slight myopia, overcome by means of 
appropriately graduated lenses 

- he has an adequate emotional adjustment. He 
evidences some insecurity when he reads aloud. 

Andres is believed to have a specific readjng disability 
related to a brain dysfunction which evidences particu- 
larly in problems of selective attention: his attention 
scatters easily, it is attracted by too many stimuJi and 
Andres is not able to concentrate on tasks that are 
relevant to finishing school work. 



23. In Andres case we observe something which is called disparity. 
If we apply Bateman's criterion, the disparity here is signifi- 
cant because 

A. there is a difference of two years between Andres* reading 
level and the level to which he should correspond. 

B. Andres I.Q. is superior and his reading performance is 
inferior 



C. Andres reading does not correspond to a fourth grade level 

D. there is a noticeable imbalance between Andres* reading 
level and his reading performance 
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2i*. The approach of chose who wrote the dei>cripcion of Andres' case is 



A. 


behavioral 


B. 


etiological 


C. 


idiopathic 


D. 


pedagogical 



25. If we analyze the description of Andres* case taking as a refer- 
ence point Bateman's definition of specific learning disabilities, 
we may conclude that 

A. it could be a specific learning disability because the 
observed discrepancy corresponds to the discrepancy which 
Bateman describes and because there are signs of brain 
dysfunction 

B. this case corresponds to a specific learning disability 
because there are no emotional disturbances 

C. this case may perhaps not correspond to a specific learning 
disability because Andres' educational background is not 
described 

D. this description may perhaps not correspond to a specific 
learning disability because there are some sensory losses 
of ci visual kind and some emotional maladjustment 

26. Andres* myopia may be considered as a problem of the following 
kind 

A. central 

B. psychoneurological 

C. peripherial 

D. determinant 

27. The fact that Andres confuses letters of similar shapes (which 
only differ in detail) makes us suppose that there is a problem 
which is fundamentally at the following level 

A. input 

B. information procession 

C. output 

D. feedback 
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28. When we say that "it is believed" that Andres has a specific 

learning disability, a definitive conclusion is not being given. 
The reason for this is that the diagnoses 

A. depend on the number of professionals who have been involved 
in the previous observations which support the diagnosis 

B. belong to the category of inferences; hence, their subjective 
component is recognized 

C. are only given in a definitive manner when the professionals 
who make them have a recognized professional status 

D. proceed from the field of medicine and, for this reason, 
must be avoided in special education 



DIRECTIONS: 

Observe the writing errors (in dictation) of C.E., a ninc-ycar-cid 

boy. He is in the third grade and receives special help because of 

reading and writing problems. Answer questions 29 and 30 (circle 
the right option). 



3,— .d/^ pA^^^<Lx. 



(The original text said: El regalo de cumpleanos. Mi 
papa me llevo de passeo. Fuimos a la montana y nos 
subimos al teleferico iQue hcrmoso regalo recibl en mi 
cumpleanos ! ) 
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The errors in dictation are evidence of some auditory distortions 
which we have underlined. Therefore, we may say that C.E. shows 
problems in the processes of 



A. input 

B* integration 

C. output 

D. feedback 



These problems seem to proceed from disturbance? of the processes 
of 



A. Input 



B. integration 

C. output 

D. feedback 




DIRECTIONS: 



Read the following summary of a historical case in the field of 
learning disabilities. 



"A boy, 12 years of aga, was brought in March 1902 to the 
Eye Infirmary by his mother, to see if there was anything 
wrong with his eyesight. The boy has been seven years at 
school, and there had been from the outset the greatest 
difficulty in teaching him to read. The boy should have 
been in Standard V, but was now, after seven years, only 
in Standard II, and he could not get out of it because of 
of his reading. He had made no complaint whatever about 
his vision, but his mother had brought him to the Eye 
Infirmary in order to discover if his eyesight had any- 
thing to do with his difficulty. His mother stated that 
he was in every other respect a sharp and intelligent 
boy. He had no difficulty with arithmetic, and cou3d 
keep up with the other students easily in this department. 
He was now working at compound addition. His mother said 
that the other boys laughed at him in class, and that 
when he became excited his reading was worse than ever. 
He concealed his defect for a time by learning his lesson 
by heart. His auditory memory, therefore, was evidently 
very good. On examining him I found that his reading 
was very defective for a boy who had been seven years in 
school. He could rarely read by sight more than two or 
three words, but came to a standstill every second or 
third word, and was unable to proceed unless he was 
allowed to spell out the word aloud, thus appealing 
to his auditory memory, or to spell it silently with 
his lips, thus appealing to his memory of speech 
movements .... He read all combinations of figures 
with the greatest fluency up to millions. I made hiip 
do several sums up to compound addition .... I 
wrote to his schoolmaster for information about the 
boy. He replied that the lad had experienced throughout 
his whole career in the school the greatest difficulty 
in learning to read, which had kept him very much behind 
in his progress through the school. He was strong in 
arithmetic, good at spelling, and average in other 
subjects, including geography and history. 'I have how- 
ever,' said the master, 'seen a case similar to this 
one in my twenty-five years' experience as a teacher. 
There is another boy in his class v;ho is quite a poor 
reader, but this other boy shows no sign of smartness 
in anything.'" (Hinshelwood , J., 1917). 
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Give three reasons why this could be a case of specific reading 
disability: 



Although there are not enough data about the second case , 
indicate which kind of case the other pupil mentioned at the 
end of the teacher's letter might probably be 

A. a nonspecific learning disa'jility 

B. a specific learning disability 

C. a general learning disability 

D. intellectual dificiency 

In order to correct educationally the learning difficulties of 
these two boys, circle in the following list the professional 
who would be best in each case 

A. primary school teacher 

B. specialist in mental retardation 

C. teacher, specialized in learning disabilities 

For pupil No. 1: 

For pupil No. 2: 

Briefly write what you believe to be specific learning 
disabilities : 



THANK YOU FOR YOUR COOPERATION. 



TABLE OF SPECIFICATIONS AND ANSWER KEY FOR THE POSTTEST 



Item Objective Level Answer Level 

1 2 A F (specific) 

2 2 A F (Spec. Ed.) 

3 3 A F (idiopathic) 

4 6 A T 

5 1-6 A F (mental) 

6 3 A F (caure) 

7 1 A K (internal) 

8 1 A T 

9 6 A F (pseudo M.R.) 

10 1-6 A F (instrumental) 

11 1 B C 

12 IBB 

13 3 A B 

14 6 A D 

15 SAB 

16 1 A C 

17 5 A C 

18 6 B B 

19 2-5 B C 

20 6 ' D 

21 1-5 A, B, D, E, F 

22 6 B D 
2J 6 B A 

24 3 B B 

25 2 B A 

26 6 B C 

27 4 B A 

28 IBB 

29 4 B C 

30 4 B A 

31 5-1 C - Significant 3 

discrepancy 

- Normal intelligence 

- Normal experience 
(st imulation) 

- Disability specific 
to one area 

- Sensorial and 
emotional integrity 

32 6 B D 2 

33 IB- D.A. specialist 2 

- R.M. specialist 
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five characteristics 
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APPENDIX B 
QUESTIONNAIRE 

This Appendix presents a translated version of the question- 
naire administered to all of the participants in the study. On 
pages 97 - 106 the questionnaire used with Groups 1 and 2 is 
presented; on pages 107-115 the questionnaire used with Group 3 
is presented. 



QUESTIONNAIRE 
(Administered to Groups 1 and 2) 



1 . Name 



2. Age 



3. Educational Level 



STUDIES PERFORMED 
(indicate the field) 


LEVEL REACHED 






Technical Studies 


a") 


^ ) 




^ ) 


^ } 


*J } 


^ ) 






Higher Educatioi, 
<>/ 


a ) 








r ^ 


A\ 
^/ 


a ^ 






Post-graduate Studies 
a) 


a) 


b) 


b) 


c) 


c) 


d) 


d) 







4.0 



How long had you not been studying before you entered U.N. A.? 
(Years and months) 



5.^) What is your present occupation? 
6.0 Do you have a paid job? 

6.2 No 



6.1 Yes 



pass to question 9.0, 



6.1.1 Indicate how many jobs you have 



6.1.2 What kinds of job(s) do you have? 



7.0 How many hours do you devote to work each we.^k? 



8.0 Do you have people under your economic responsibility? 

8.1 Yes j I 8.2 I t > pass to question 9.0. 



8.1.1 Indicate the number of persons who depend on 
you 

9.0 How many sessions did you need to study the material we gave 
you? (indicate with an X). 



Number of 
Sessions 



2 - 3 



5 6 



8 



10 more 



□□□□□ jnnnnn 



10.0 How long in total did it take you approximately to study the 
material? (indicate hours and minutes) 



11.0 How did you study the material? Did you just read it (in one 
or several sessions) and then go on to the present meeting? 

11.1 Yes I \ — — > pafis to question 12.0. 

11.2 No 



V 



Pass to next page. 



Indicate how you studied: 





Very 
Often 


r requen L ly 


oomc t imes 


Rarely 


li c V c L 


1, You reread the 
main parts 












2, You took notes 












3. You drew graphs 












A, You made 
abstracts 












5. You underlined 
what seemed 
important to 
you 












6. You answered 
every question 












7. You compared 
your answers 
with the 
answers in 
the text 












8. You used a 
Spanish dic- 
tionary to 
clarify some 
doubts 












9. You used a 
specialized 
dictionary to 
clarify some 
doubts 












10. You read the 
additional 
bibliography 












11. You tried to 
get additional 
bibliography 
and could not 
get any 












12. You followed 
the indicated 
sequence 












13. You changed the 
sequence of the 
material 













in 





Very 
Often 


Frequently 


Sometimes 


Rarely 


Never 


lA, You asked a 

fellow student 
or a friend for 
help 












15. You asked a 

specialist for 
help 












16. You sought 

counseling at 
the Local 
Center 















ii2 



12.0 In order to learn this material, did you review? 



12.1 Yes 12,2 Ho | > pass to question 13.0 



12.1.1 How many times? 



12.1.2 How long approximately? 

12.1.3 Indicate when: 



before each study* session? 
before some study sessions? 
after the study sessions? 
before the present meeting? 



13,0 Were you interested by the material? 

13.1 Yes I I 

13.2 No I I 

13.3 Explain why 



1^4,0 Did you have any information about leorning disabilities before 
studying the material? 



14,1 Yes I I 
1A,2 No I I 

15,0 Did you acquire some new knowledge from the material? 
15,1 Yes I I 15,2 No | | 

15.3 Explain 



16,0 Do you think that you learned the material we gave you? 

16,1 Yes 16.2 No > pass to question 17.0, 



Pass to next page, 
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16.1.1 If you answered affirmatively, do you believe that any of the 
following aspects of the material may have made your learning 
easier? (indicate with an X) 





Much 


oomewna c 


(nol ac ail 


1. Introduction 








2. Objectives 








3. Organization of the text 








A. Explanations inserted in 
Lne lcXl 








J • CAcilupJLcb OI Lilc LcXL 
















/ • mibwcJLb gxvun lo lHc cy\,erciScS 








8. Style of the text 








9. Illustrations 








10. Additional comments 








11. All the above marked aspec-S 
as a whole 









16.1.2 If none of the above aspects made your learning easier, and if 
you considered that other aspects did, indicate which aspects 
they were 



17.0 V/ould you have preferred the information to have a different 
presentation? 



17.1 Yes I I 17.2 No 

17.1.1 Explain how 



pass to question 18.0. 
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18,0 Did you hav^ any doubts or confusion during your study of the 
material? 



18.1 Yes I I 18.2 No [ | > pass to question 19.0. 



V 



18.1.1 Wi.at were they? 



18.1.2 If you could solve them, how did you do so? 



18.1.3 If you could not, explain why. 



19.0 Do you think that something may be missing in the study material 
you received? 



19.1 Yes 



T 



19 . 2 No 



-> . jss to question 20.0. 



19.1.1 Explain 



20.0 Is there any specific aspect of ch.is material chat you especially 
liked? 



20.1 Yes I I 20.2 No | | 20.3 Expl 



am 



21.0 Is there any specific aspect of che material that you disliked? 
21.1 Yes I I 2.12 No | | 21.3 Explain 
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22.0 Do you believe there is any difference between the study 

material we gave you for the present research and the texts 
produced by U.N.A. that you already know? 



22.1 Yes 



22.2 No I ] -> pass to question 23.0. 



22.1.1 Do you consider that the difference 
facilitates learning? 



22.1.2 Explain how this material influenced your 
learning. 



23.0 Do you believe that there is any difference between the 

instructional material you received for the present research 
and a common textbook? 



23.1 Yes i 23.2 No | > pass to question 2A.0 



23.1.1 Explain how the difference affected your 
learning. 



24.0 Did the kind of exercises included in the material influence 
your learning? 

2A.1 Yes I I 2A.2 No {J^H 

2A.3 Explain how it affected your learning. 



25.0 With regard to the exercises interspersed in the text, 
25.1 Indicate some positive aspects they might have 



25.2 Indicate some negative aspects they might have 



1)6 



105 



26.0 Did you look for additional information about learning 
disabilities after studying the material we gave you? 



26^1 Yes [ I 26.2 No | \ > pass to question 27.0. 

26, 1. ) Explain where you looked for information? 

1. In Books 

2. In reviews 

3. With advanced students 

4. Other students 

5. A counselor or an adviser 

6. Other sources 



27.0 Do you want to know more about learning disabilities after 
studying this material? 

27.1 Yes I I 27.2 No [ | 

27.3 Why? 

28.0 Did you find that the material took a long time to stud>'^ 
28.1 Yes I I 28.2 No | | 
28.3 UTiy? 



29.0 Did you ever desire to drop out of the study? 

— > pass to question 30.0. 



29.1 Yes -29.2 No 



29.1.1 What induced you to carry on with this 
research? 



30.0 Indicat'j '^htee positive and three negative aspects of the 
inscructional material we gave you. 
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31.0 Is there any additional comment on the instructional material 
that you want to make? 



ERIC 
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Name 



QUESTIONNAIRE 
(.Administered to Group 3) 



Age 

Educational Leval 



STUDIES PERFORMED 
(indicate the field) 


LEVEL REACHED 


High School 




Technical Studies 
a) 


a) 


b) 


b) 


c) 


c) 


d) 


d) 






Higher Education 
a) 


a) 


b) 


b) 


c) 


c) 


d) 


d) 






Post-graduate Studies 
a) 


a) 


b) 


b) 


c) 


c) 


d) 









4,0 How long had you not been studying before you entared U.N. A.? 

(Years and months) 

5.0 What is your present occupation? 
6.0 Do you have a paid job? 

6.2 No 



6.1 Yes 



->pass to question 9.0. 



6.1.1 Indicate how many jobs you have 



6.1.2 What kinds of job(s) do you have? 



7.0 How many hours do you devote to work each week: 



8.0 Do you have people under your economic responsibility? 

8.1 Yes { I 8.2 | [ -> pass to question 9.0. 



V 

8.1.1 Indicate the number of persons who depenu on 
you 

9.0 How many sessions did you need to study the material we gave 
you? (indicate with an X). 

Number of 

Sessions 12 3456789 10 more 

□□□□□□□□□□□ 

10.0 How long in total did it take you approximately to study the 
material? (indicate hours and minutes) 



11.0 How did you study the material? Did you just read it (in one 
or several sessions) and then go on to the present meeting? 

11.1 Yes I I > pass to question 12.0. 

11.2 No r~] 



V 



Pass to next page. 



12{) 



Indicate how yu^ studied: 





Very 
Often 


Frequently 


Sometimes 


Rarely 


Never 


1. You reread the 
main parts 












2 . You took notes 













3. You drew graphs 












A. You made 
abstracts 












5. You underlined 
what seemed 

lmnnr^an^ tn 

you 












6. You answered 
every question 












7. You compared 
your answers 
with the 
answers in 
the text 












8. You used a 
Spanish dic- 
tionary to 
clarify some 
doubts 












9. You used a 
specialized 
dictionary to 
clarify soine 
douh t 

\>t Cr U fc> O 












10. You read the 
additional 
bibliography 












11. You tried to 
get additional 
bibliography 
and could not 

opt* jjnv 












12. You followed 
the indicated 
sequence 












13. You changed the 
sequence of the 
material 
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Very 
Often 


Frequently 


Sometimes 


Rarely 


Never 


14. You asked a 

fellow student 
or a friend for 
help 












15. You asked a 

specialist for 
help 












16 • You sought 

counseling at 
the Local 
Center 
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12.0 In order to learn this material, did you review? 

12.2 No 



12.1 Yes 



r 

12.1.1 How many times? 



> pass to question 13.0, 



12.1.2 How long approximately? 

12.1.3 Indicate when: 



- before each study session? 

- before some study sessions? 

- after the study sessions? 

- before the present meeting? 



13.0 Were you interested by the material? 

13.1 Yes I 1 

13.2 No I I 

13.3 Explain why 



lA.O Did you have any information about learning disabilities before 
studying the material? 



U.l -es I I 



14.2 No [ 



15.0 Did you acquire some new knowledge from the material? 

U 15.2 No I I 



15.1 Yes 



15.3 Explain 



16.0 Do you think that you learned the material we gave you? 
16.1 Yes 



16.2 No 



pass to question 17.0. 



V 



Pass to next page. 
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16,1.1 If you answered affirmatively, do you believe that any of the 
following aspects of the material may have made your learning 
easier? (indicate with an X) 





Much 


Somewhat 


Not at all 


1. Introduction 








2. Objectives 








3, Organization of the text 








A. Explanations inserted in 
the text 








5. Examples of the text 








6. Exercises 








7. Answers given to the exercises 








8. Style of the text 








9. Illustrations 








10 • Additional comments 








11. All the above marked aspects 
as a whole 









16.1.2 If none of the above aspects made your learning easier, and if 
you considered that other aspects did, indicate which aspects 
they were 



17.0 Would you have preferred the information to have a different 
presentation? 

17.1 Yes I I 17.2 No [ | > pass to question 18.0. 

17.1.1 Explain hov; 



113 



18,0 Did you have any doubts or confusion during your study of the 
material? 



18,1 Yes I I 18,2 No | | > pass to question 19-0. 



r 



18*1.1 What v;ere they? 



18.1.2 If you could solve them, how did you do so? 



18,1.3 If you could not> explain why. 



19,0 Do you think that something may be missing in the study material 
you received? 



19,1 Yes I I 19,2 No | | > pass to question 20,0, 



19.1,1 Explain 



20,0 Is there any specific aspect of this material that you -^specially 
liked? 



20.1 Yes I I 20,2 No j | 20,3 Explai 



21,0 Is there any spacific aspect of the material that you disliked? 
21,1 Yes I I 2,12 No | | 21,3 Explain 



ERLC 
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22.0 Do you believe there is any difference between the study 

material we gave you for the present research and the texts 
produced by U.N. A. that you .already know? 



22.1 Yes 



22.2 No [ I > pass to question 23.0, 



22.1.1 Do you consider that the difference 
facilitates learning? 



22.1.2 Explain how this material influenced your 
learning- 



23.0 Do you consider that there is any difference between the 

study material you received for the present research and a 
common textbook? 



23.1 Yes 



23.2 No 



pass to question 24.0. 



23.1.1 Explain how the difference affected your 
learning. 



24.0 Did you look for additional information about learning 

disabilities after studying the material you received from us? 



24.1 Yes 



24.2 No I I > pass to question 25.0. 



24.1.1 Explain where you looked for information: 



1. In books 

2. In reviews 

3. With advanced students 

4. Other students 

5. A counselor or an adviser 

6. Other sources 



25.0 Do you want to get more information about learning disabilities 
after studying this material? 



25.1 Yes I I 25.2 No | | 
25.3 Why? 
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26»0 Did you find it took a long time to study the material? 
26*1 Yes 1 I 26.2 No | | 

26-3 Why? 



27.0 Did you ever desire to drop out of the study? 

27.1 Yes I 27.2 No | | > pass to question 28.0, 



r 



27.1.1 \n\at induced you to continue in the research? 



28.0 Indicate three positive and three negative aspects of the 
instructional material you studied, 



29.0 Is there any additional comment on the instruction'", material 
that you want to make? 
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APPENDIX C 
INSTRUCTIONS AND INFORMATION 

This appendix presents In translation two pieces 
of information given to the participants in this study. 



ITT 

Appendix C.l 
INTRODUCTION 

The National Open University is an ambitious innovation in 
higher education. In its aspiration to reach an optimum level of 
academic excellence, the university considers that research, 
especially educational research, should be one of its priorities. 

At the present, the area of Education is carrying out a 
research study on instructional materials for distance education. 
Among all the students in the area of Education, we selected the 
candidates in the field of Learning Disabilities to ask them to 
cooperate in the above mentioned research. 

If you decide to give us your help, it will consist in studying 
instructional material thai we will give you as soon as you give us 
an affirmative response. The maximum time required to perform this 
task is approximately 20 hours (although it could be much less). 
You will take the material home with you and study it in your free 
time. After this, you will return to this same place, on 

October the th, at 8:30 a.m. for a meeting that will take two 

hours at the most. 

The cooperation we are asking you to give us is fundamental for 
the achievement of the present research. We hope that the results 
will allow us to draw conclusions that will improve our instructional 
approach and, hence, the educational support that we will be giving 
you in the future. 

If you agree to participate, let us know now. If you do not, 
please leave this text on the table. 
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We thank you very much for your cooperation, and hope that this 
experience, if you decide to participate, will contribute to the 
education we wish to offer you in the field of Learning Disabilities* 
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Appendix C.2 
INFOR>IATION 

You are now entering the second phase of the research you are 
involved in and we believe that it is a good .aoment to enlarge your 
understanding of the activity in which you are participating. 

You have probably observed that not all your fellow members 
received the same study material. The reason for this is that we 
intend to measure the effects of different instructional materials, 
in order to draw useful conclusions which will be of help in the 
elaboration of our future books. 

The task we are going to ask you to follow now is simple. 
First, we will ask your opinion about several subjects related to the 
employment of the material you have had to study. Next, you will 
receive a questionnaire about the contents of the material you have 
studied. We hope that you answer as sincerely as possible since we 
are looking for information on the group reaction as a whole, and net 
the individual grades each one of you is going to obtain. 

This research has a third and last phase. In order to carry out 
this phase, we are asking you to return to the Liceo Gustavo Herrera, 
next month (on November, the 27th, Saturday) at8:30 a.m., for a 
meeting similar to the present one; iL will be of approximately one 
hour. 

At the end of this work session, we will be at your dis. "^sition 
for any information you may wish to obtain about the career of Learning 
Disabilities. Finally, we again thank you very much for your 
cooperation. 



APPENDIX D 

- Some methological remarks on the study 
(by R. Schuemer) 

- The author's reply (Anne Benko de Rotaeche) 



R. Schuemer: 

Some methodological remarks on the study 



(1) in my opinion the main difficulty in testing hypotheses derived 
from cognitive (instructional) theories seems to be that many 
concepts and terms from the field of cognitive theories (e.g. 
hierarchical structures, 'depth of cognitive processes', scheme 
and concept learning, 'inspection res>pectively mathemayenic 
behaviors' etc.) certainly are illustrative but also quite 
imprecise. To me above all it seems that clear operational 
definitions are missing in this field (compare, for example, 
the quotation of Rumelhart & Ortony on p. 28). 

(2) The subject of the empirical part of the investigation is the 
evaluation of three didactical instruction conditions or versions. 
Hypothesis 1 and the test made for it (univariate variance analysis) 
imply an overall comparison of the achie^^en^ents of the posttest 
following the completion of the course (the latter is developed in 
accordance with the respective instructional condition). 

Such overall tests produce some interpretational nroblems: 

- Instructional conditions usually consist of various didactical 
elements or can be interpreted as combinations of such elements, 
but only the global effect of the instructional conditions resp. 
versions can be tested. Therefore it remains unsolved which 
individual components of the instruction contribute to the 
hypothesized better achievement under certain instructional 
conditions. So, even if the achievements are better after a 
certain type of instruction than after the others, one does not 
know which component of the instruction has lead to that 
improvement. 
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- Instructional/didactical elements (as, for example, 'stimulation 
elements' or 'illustrations') cannot be regarded without 
simultaneously considering the instructional contents; what may 
hava an positive effect in connection with one certain subjecv. 
matter may be absolutely ineffectual (or even troublesome) in 
another(impossibility of generalization). 

- A didactical treatment of instructional texts may also lead to 
alterations of contents. To insert exercises or 'advance 
organizers' in an instructional text, for example, seems to change 
it least the redundancy of the text and may possibly even 

influence the structure of the contents. ' This may also be 
valid for the stressing of certain facts, points or parts of the 
text which puts other parts in the background. The question 
therefore is whether (possible) differences in achievement after 
the completion of courses with different didactical treatment 
are caused purely by the didactical treatment or also by 
- the thereby changed - contents (problem of confounding). 

- With regard to this problem it is not clear to me why for the 
second (separated) - and also for the third (eliminated) - 
version/instructional condition another text has been used 
than for the first condition ('interspersed'). Admittedly, 

it is said on p. 42 that the material 'corresponds in content to 
the "instructional text" of the first version', but still it 
cannot be excluded that it differs from the first versions not 
only in the didactical presentation but also in contents. 

The above indications of some problems which (often) occur in 
studies of this kind should not really be taken as criticism ' ^ 
but as a suggestion for readers or 'consumers' of such evaluation 
studies to bear in mind the limits of the investigation's 
approach. 



1) The cognitive structuring of contents by the learner left aside. 

2) If it is criticism at all, then it is also self-criticism: 

the author was co-author of a study which was carried out on the 
basis of exactly the same approach (cf. Holmberg, Schuemer & 
Obermeier 1982). 



(3) Finally some rather technical details: 

(a) Design (pp. 49 - 50) 

Good arguments can be cited as well for as against 
'posttest only* designs; the same could be said regarding 
'pretest/posttest* designs. The advantage of the 'posttest 
only' design is that possible 'facilitative effects' (or 
other similar effects produced by repeated measurement) can 
be avoided; on the other hard problems may result in this 
type of design if 

- there are great individual differences between subjects 
before treatment (the test of hypotheses then will not be 
very efficient) 

- the groups of subjects investigated under each instructional 
condition are very small (only 14 (!) under each condition 
in the test of the main hypotheses 1 - see p. 48) 

and/or if 

- there is reason to assume that the groups under the various 
conditions differ in some systematical way. 

The 'pretest/posttest' design may help to avoid some of these 
problems: one can analyse the differences between posttest and 
pretest scores thereby abstracting interindividual differences 
before treatment (or one can use the pretest scores as a 
covariate in an analysis of covaricnce). 

A combination of both design types would be optimal, for 
example as fellows: 



group 1 


group 2 




pretest 


treatment 


treatment 


posttest 


posttest 



(b) Operational definitions of the dependent variables 

It is not clear to me how the posttest score is defined 
exactly - for example: Is it the number of correct answers? 
(Then a non-parametric method of analysis - e.g. the H-test 
of 'Kruskal-Wallis or another homologous test - would seem 
to be preferable. ) 

Also the (operational) definition of the three 'levels of 
processing' for hypotheses 3 is not clear to me (compare 
also pp. 8-9 'dependent variables'). 

To make use of an analysis of variance for the test of hypothesis 
seems to me - to express it cautiously - somehow doubtful. The 
method used is a uni variate analysis of variance with 
two independent factors: = treatment (i.e. version) and 

= processing level. 
The dependent variable is the achievement; since the latter is 
assessed on the different levels of F^ by differing items in the 
posttest, not just one dependent variable but three different 
dependent variables are analysed; this seems to drift towards 
a comparison between apples and pears. The 'highly significant' 
main effect of the 'processing level' (p. 65) may not reflect 
anything but the considerably varying numbers of items for the 
three processing levels (cf. Tabc on p. 95); fot similar reasons 
it is hardly possible to interpret the interaction. 

(c) Expectation 2 really is a treatment x learner variables) 
interaction hypothesis, which - theoretically seen - absolutely 
makes sense. Since the learner variables (age, background etc.) 
are classification variables and therefore cannot be set or 
changed arbitrarily by the researcher unequal cell frequencies 
will probably occur. This non-orthogenal i ty of design may cause 
difficulties in interpreting the F-test for the interaction 
term. There is an indication (on pp. 59 - 64) of such problems 
having occurred in this case; regrettably it is only mentioned 
that there are unequal cell frequencies, but the N per cell 

in the analyses of variance is not stated there (Tab. 7 - 11). 



Furthermore, the F-tests for the 'treatment* main effect 
differ between each other in the analyses for the hypotheses 
2a - 2c These irritating results may be caused by each 
effect being adjusted for the other effects in the design; 
nothing is said about this, however. 



1) And all these tests differ irrom the analogous test in the 
analysis for hypotheses 1 (p. 52), 
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The author's reply 

In a letter of July 1987 Anne Benko de Rotaeche commented on 
R. Schuemer's methodological remarks. With regard to the first remark 
she writes: "I recognize it surprised me: I was not aware of these lacks 
of operational definitions ... perhaps because I adopted the general 
trend in the literature."^^ 

Her answers to the other remarks give further information about the methods 
and variables of the study and are therefore reproduces in total below: 

(2) 

- Effects of the individual components of instruction: I agree that' 
this effect remained untested (p. 73), but I could not find any technical 
procedure to elucidate it, in spite of the fact it was one of my principal 
concerns at the beginning. 

- Relationship between the didactical elements and the content: I am 
aware of the strong relationship that should exist between the didactical 
procedures and the content. I really think that content - and what the 
student already knows about it, when this can be assessed - determines or 
indicates the kind of instructional strategies that might be more effective. 

In the content of the study - which was an introductory definition of 
learning difficulties - emphasis was placed on differentiating the basic 
concepts from similar ones. When the concepts were more related to abstract 
ideas, the strategies tended to be more verbal than those for "concrete" 
concepts. They consisted in: 

- presenting and having the student analyze verbal definitions in 
their component parts 

- examining verbal examples and counterexamples in their component elements 

- placing the concept in different contexts in which they could or could 
not be used 

- mi \g "maps" or schemata or classification tables in order tc summarize 
in a more visual and concrete manner 



1) Apparently^ R. Schuemer uses the term •operational definition* in a 
more restricted way. 



Furthermore, since it is widely admitted (and my own teaching experience 
reaffirms this) that, in general, Venezuelan students are quite passive in 
their learning, weak in language areas and unaware of their learning 
processes, the instructional material aimed at developing "metacogni ti ve" 
strategies in the student asking him to carry out the tasks by himself, 
to compare his answer with other possible ones given in the text, to 
apply it to other situations and to summarize the procedure employed to 
solve the problem. So, the instructional purpose actually was not only to 
have the student recognize the concept and its limits but also the 
strategies used to grasp it. And, as can be soen, the strategies are very 
much oriented toward "reading techniques". 

When the concepts were more concrete, such as defining a writing difficulty, 
in addition to all or some of the means above described, more concrete 
means were used as illustrations (a page written by a l.d. child) as an exar^ 
or an exercise for analysis. In the case of reading diff iculties, 
articulation and rythm difficulties were reproduced in a written manner 
(in the real material, an audiocassette is also available and several 
exercises are based upon it). In my opinion, a cassette or any written 
didactical means can better be used if the kind of content is taken into 
account. 



Probably, page 73 of the conclusiosn leads one to believe that I generalize 
upon the virtues of instructional treatments. I think that a phrase - that 
I recently wrote in a Spanish resume of the thesis - should be added before 
the ending of the paragraph: "There is probably a direct relationship 
between the kind of content and the text presentation; this relationship 
should be studied." 



" Didactical treatment/alterations of contents; This is a question I 
also asked myself. In order to avoid dissimilarity, a chart was made 
indicating each content item of version 1; then, each item was looked for 
in the original sources; at the same time an attempt was made to match 
the preciseness and the depth of tha treatment of the information. Therefore, 
no content item of version 1 is absent in versions 2 and 3, 

An effort was made (in version 1) to say the same thing the original author 
was saying (X is so and so), but in a different manner. For example, when 
defining an idea, the definitions of several authors were reproduced. 
Each one was anlyzed and then compared in its component elements and 



significance and then compared with the others; this led to categorizing 
the definitions, ^ In Version 3 - "eliminated" - the same authors were 
takeni but in their integral text. Students were absolutely free to 
isolate the definitions* to analyze and to compare them, if they wished to. 
.So, an important task for them should have been to realize that concepts 
they were reading about should have been clear to them. In version 2, by 
means of the Study Guidet students were invited to carry out the same 
operations as students in version 1, 



Therefore, I anree that there is a change in stresses, redundancy and 
structure of the content under study and also in the language, which tried 
to be simpler. Relationships that were not mentioned or stressed in the 
Selected Readings were established in versions 1 and 2 by means of exercises 
and text organization. But I am not sure that the content was different; 
our aim was to help students to understand the meaning of what each author 
was saying* 

In order to give an idea of how the content was reorganized in version 1, here 
the index of vers"^on 1 and of the Selected Readings, Let us compare them: 



Version 1 

- Presentation 

General Introduction 
(it t general schema - an 
"advanced organizer" - integrating 
Texts 1 and 4) 

- Unit » 

- pp, 9-24 

- pp, 25-33 

- pp; 34-37 

- Objectives 

- Self-evaluation 

- Additional comments 



Selected Readings 

Idem 
Missing 



Corresponded to Texts 1 
and 3 

" Text 2 

" Text 4 

Missing 
Missing 

Correspon«»ed to Texts 5 
and 6 



1) Kind<" of definitions: This was not a really new content item since 
the Selected Reading had a text identifying categories of l.d and 
giving one or two examples. The differences were that the analysis was 
missing, the examples were fewer and the explanations about them less 
detailed. 
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- Use of another text for versions 2 and 3; I agree that in spite of 
all the cares taken, it cannot be excluded that the versions differed 
in content. 



The reason why different texts were used is not explained in the report. 
Indeed, it was because, at that time, a discussion was going on in our 
university. Some argued that it was not necessary for distance education 
to prepare its own material since the original sources plus a study guide 
(not as analytic as the one used in the study) could be enough. I believed, 
and I still do, that special text elaboration is needed when a combination 
of the following conditions is prestent: 

- original sources are at variance with one another (no integration of 
t> trends can be found in the literature about the subject matter) 

- the language of the original source is too sophisticated and technical 

- the subject matter is taught at the lower levels of a cereer 

- students have weak study habits and reading techniques, and live in poor 
socio-cultural environments, as is the case in Latinamerican provinces. 

The unit under study was elaborated to create a cognitive structure in 
which to anchor the content that would follow in the next units. And the 
whole course also had this purpose: to construct a network of basic contents 
to which the next courses - less directive than those within the 
curriculum - could refer to. 



(3) 



(a) Desigr; I thank you for your suggestion for a better design. I should 
have made it clear that the intention was also to avoid that group 3 should 
get information about our objectives. 

With regard to the dimension of the sample, this was a problem easy to 
predict but difficult to avoid. Drop out was a predictable problem, but to 
avoid it would have demanded some kind of support from the University as 
an institution: for example, to offer the different versions of the unit as 
regular units (so that it would have been obligatory for everyone) or to 
give some compensation for the cooperation (credits, or instructional 
material), but both kinds of support were impossible to obtain in a short 
span of time. 
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(b) Operational definitions of the dependent variables: 

- Definition of the posttest scores: The scores differed according 
to the complexity of the required answer: 

- Level A and B: number of correct answers. 

- Level C: naif of the scores were given to the number of correct 

details mentioned and half of the scores were given to 
the quality of the integration. 

- Definition of the three levels of processing: In effect, definitions 
on pp. 8-9 are not operational enough. The criterion used to finally 
classify an item at each level was: 

(1) Given several options, the student had to select \.'hich one was 
the best 

- to identify a concept, a fact or a principle: Level A. 

- to translate a piece of information from one language 
into another: Level B. 

- to argue a point of view basing the choice upon the theoretical 
content given in the unit: Level B. 

(2) Given a question, the student had to write a personal synthesis 
about the subject matter asked, demonstrating a coherent 
integration of ideas and a correct use of concepts. 

I think you are right about the three dependent variables. 

(c) Effectively, unequal cell frequency did occur. Data about the 
distribution are available. 
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